[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:12:19 dist-test-slave-6k22 krb5kdc[18720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:12:19 dist-test-slave-6k22 krb5kdc[18720](info): set up 2 sockets
Aug 06 02:12:19 dist-test-slave-6k22 krb5kdc[18720](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 02:12:20.897428 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:12:20 dist-test-slave-6k22 krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:12:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:12:21.027117 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:39021
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:34943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:39021
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:21.167923 18733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:21.168236 18733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:21.168318 18733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:21.172220 18733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:21.172298 18733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:21.172334 18733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:21.172387 18733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:21.172425 18733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:21.177752 18733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:34943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:39021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:39021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:21.178942 18733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:21.180141 18733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:21.191823 18741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:21.191833 18739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:21.192028 18738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:21.192188 18733 server_base.cc:1034] running on GCE node
I20260806 02:12:21.192773 18733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:21.194270 18733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:21.195466 18733 hybrid_clock.cc:648] HybridClock initialized: now 1785982341195429 us; error 47 us; skew 500 ppm
Aug 06 02:12:21 dist-test-slave-6k22 krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982341, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:21.198285 18733 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:12:21.199222 18733 webserver.cc:460] Webserver started at http://127.18.70.126:41697/ using document root <none> and password file <none>
I20260806 02:12:21.199858 18733 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:21.199954 18733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:21.200278 18733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:21.202306 18733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "09c2415ba5ea42e3aaaec1a7c7fdb865"
format_stamp: "Formatted at 2026-08-06 02:12:21 on dist-test-slave-6k22"
server_key: "a281fa9687fad23eb8fbc4b965567d36"
server_key_iv: "1a1cd05dca519509c093ceae483beb6f"
server_key_version: "encryptionkey@0"
I20260806 02:12:21.202862 18733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "09c2415ba5ea42e3aaaec1a7c7fdb865"
format_stamp: "Formatted at 2026-08-06 02:12:21 on dist-test-slave-6k22"
server_key: "a281fa9687fad23eb8fbc4b965567d36"
server_key_iv: "1a1cd05dca519509c093ceae483beb6f"
server_key_version: "encryptionkey@0"
I20260806 02:12:21.206890 18733 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 02:12:21.209455 18748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:21.210796 18733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 02:12:21.210934 18733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "09c2415ba5ea42e3aaaec1a7c7fdb865"
format_stamp: "Formatted at 2026-08-06 02:12:21 on dist-test-slave-6k22"
server_key: "a281fa9687fad23eb8fbc4b965567d36"
server_key_iv: "1a1cd05dca519509c093ceae483beb6f"
server_key_version: "encryptionkey@0"
I20260806 02:12:21.211047 18733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:21.222178 18733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:21.225405 18733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:21.225634 18733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:21.233382 18733 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:39021
I20260806 02:12:21.233436 18803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:39021 every 8 connection(s)
I20260806 02:12:21.234413 18733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:21.237978 18804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:21.244081 18804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865: Bootstrap starting.
I20260806 02:12:21.244484 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 18733
I20260806 02:12:21.244647 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:21.245970 18713 external_mini_cluster.cc:1428] Setting key 88abd0bcadd0f81492d1ee934f7c571c
I20260806 02:12:21.246716 18804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:21.247510 18804 log.cc:824] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:21.249763 18804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865: No bootstrap required, opened a new log
I20260806 02:12:21.254025 18804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09c2415ba5ea42e3aaaec1a7c7fdb865" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39021 } }
I20260806 02:12:21.254289 18804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:21.254374 18804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09c2415ba5ea42e3aaaec1a7c7fdb865, State: Initialized, Role: FOLLOWER
I20260806 02:12:21.254838 18804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09c2415ba5ea42e3aaaec1a7c7fdb865" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39021 } }
I20260806 02:12:21.255028 18804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:21.255095 18804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:21.255191 18804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:21.256104 18804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09c2415ba5ea42e3aaaec1a7c7fdb865" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39021 } }
I20260806 02:12:21.256479 18804 leader_election.cc:302] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09c2415ba5ea42e3aaaec1a7c7fdb865; no voters:
I20260806 02:12:21.256807 18804 leader_election.cc:288] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:21.256914 18809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:21.257136 18809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [term 1 LEADER]: Becoming Leader. State: Replica: 09c2415ba5ea42e3aaaec1a7c7fdb865, State: Running, Role: LEADER
I20260806 02:12:21.257431 18809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09c2415ba5ea42e3aaaec1a7c7fdb865" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39021 } }
I20260806 02:12:21.258210 18804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:21.258848 18809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09c2415ba5ea42e3aaaec1a7c7fdb865. Latest consensus state: current_term: 1 leader_uuid: "09c2415ba5ea42e3aaaec1a7c7fdb865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09c2415ba5ea42e3aaaec1a7c7fdb865" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39021 } } }
I20260806 02:12:21.258967 18809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:21.259480 18813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:21.259548 18810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09c2415ba5ea42e3aaaec1a7c7fdb865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09c2415ba5ea42e3aaaec1a7c7fdb865" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39021 } } }
I20260806 02:12:21.259789 18810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:21.263391 18813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 02:12:21 dist-test-slave-6k22 krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:21.269599 18813 catalog_manager.cc:1347] Generated new cluster ID: f38ace01e4a34c1aaa10306e5278049e
I20260806 02:12:21.269658 18813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:21.272563 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:21.249730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32968 (local address 127.18.70.126:39021)
0806 02:12:21.250233 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:21.250253 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:21.251433 (+ 1180us) server_negotiation.cc:407] Connection header received
0806 02:12:21.254012 (+ 2579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:21.254030 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:21.254448 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:21.254811 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:21.257273 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:21.258567 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:21.260336 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:21.261060 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:21.261232 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:21.266218 (+ 4986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:21.266225 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:21.266241 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:21.266272 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:21.268469 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:21.269139 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:21.269146 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:21.269152 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:21.269303 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:21.269799 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:21.269803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:21.269804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:21.270306 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:21.270565 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:21.271157 (+ 592us) server_negotiation.cc:299] Negotiation successful
0806 02:12:21.271442 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":137,"threads_started":1}
I20260806 02:12:21.300220 18813 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:21.301576 18813 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:21.309731 18813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09c2415ba5ea42e3aaaec1a7c7fdb865: Generated new TSK 0
I20260806 02:12:21.310662 18813 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:12:21.351272 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:39021
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:34943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:21.478252 18830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:21.478538 18830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:21.478626 18830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:21.483212 18830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:21.483268 18830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:21.483372 18830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:12:21.488557 18830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:34943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:39021
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:21.489606 18830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:21.490633 18830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:21.500859 18835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:21.501034 18830 server_base.cc:1034] running on GCE node
W20260806 02:12:21.500877 18836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:21.500877 18838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:21.501536 18830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:21.502223 18830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:21.503432 18830 hybrid_clock.cc:648] HybridClock initialized: now 1785982341503417 us; error 28 us; skew 500 ppm
Aug 06 02:12:21 dist-test-slave-6k22 krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982341, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:21.506316 18830 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:12:21.507098 18830 webserver.cc:460] Webserver started at http://127.18.70.65:39105/ using document root <none> and password file <none>
I20260806 02:12:21.507673 18830 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:21.507742 18830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:21.507915 18830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:21.509820 18830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "eaf425336bfa4ea995a6fbfcac03150f"
format_stamp: "Formatted at 2026-08-06 02:12:21 on dist-test-slave-6k22"
server_key: "08246c85ab5afa36bda4a98969c67b07"
server_key_iv: "b5665212ae07c1fcfaa2649972c900af"
server_key_version: "encryptionkey@0"
I20260806 02:12:21.510325 18830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "eaf425336bfa4ea995a6fbfcac03150f"
format_stamp: "Formatted at 2026-08-06 02:12:21 on dist-test-slave-6k22"
server_key: "08246c85ab5afa36bda4a98969c67b07"
server_key_iv: "b5665212ae07c1fcfaa2649972c900af"
server_key_version: "encryptionkey@0"
I20260806 02:12:21.514163 18830 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 02:12:21.516606 18845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:21.517895 18830 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:21.517997 18830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "eaf425336bfa4ea995a6fbfcac03150f"
format_stamp: "Formatted at 2026-08-06 02:12:21 on dist-test-slave-6k22"
server_key: "08246c85ab5afa36bda4a98969c67b07"
server_key_iv: "b5665212ae07c1fcfaa2649972c900af"
server_key_version: "encryptionkey@0"
I20260806 02:12:21.518079 18830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:21.550203 18830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:21.553953 18830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:21.554116 18830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:21.554697 18830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:21.555509 18830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:21.555579 18830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:21.555622 18830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:21.555646 18830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:21.565580 18830 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:45411
I20260806 02:12:21.565615 18962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:45411 every 8 connection(s)
I20260806 02:12:21.566463 18830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:12:21.567966 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 18830
I20260806 02:12:21.568054 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:12:21.568346 18713 external_mini_cluster.cc:1428] Setting key 220e46af8170d01c978e83a343ec512d
Aug 06 02:12:21 dist-test-slave-6k22 krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982341, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
I20260806 02:12:21.581260 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:21.568465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:52061 (local address 127.18.70.126:39021)
0806 02:12:21.568626 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:21.568631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:21.569624 (+ 993us) server_negotiation.cc:407] Connection header received
0806 02:12:21.571243 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:21.571246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:21.571300 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:21.571384 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:21.572646 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:21.573183 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:21.574323 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:21.574557 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:21.574669 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:21.577690 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:21.577694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:21.577697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:21.577726 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:21.579364 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:21.579898 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:21.579901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:21.579902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:21.579975 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:21.580305 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:21.580308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:21.580309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:21.580477 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:21.580551 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:21.581025 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 02:12:21.581136 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
I20260806 02:12:21.582266 18965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:21.568977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:39021 (local address 127.18.70.65:52061)
0806 02:12:21.569480 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:21.569532 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:21.571032 (+ 1500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:21.571599 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:21.571608 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:21.571990 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:21.572499 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:21.572514 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:21.573314 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:21.573317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:21.574196 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:21.574205 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:21.574716 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:21.574720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:21.574963 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:21.575800 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:21.575826 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:21.577504 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:21.579483 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:21.579488 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:21.579500 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:21.579808 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:21.580057 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:21.580060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:21.580061 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:21.580194 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:21.580570 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:21.580575 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:21.580812 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 02:12:21.580964 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 02:12:21.581213 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":145,"threads_started":1}
I20260806 02:12:21.583797 18963 heartbeater.cc:347] Connected to a master server at 127.18.70.126:39021
I20260806 02:12:21.584061 18963 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:21.584553 18963 heartbeater.cc:511] Master 127.18.70.126:39021 requested a full tablet report, sending...
I20260806 02:12:21.586790 18766 ts_manager.cc:194] Registered new tserver with Master: eaf425336bfa4ea995a6fbfcac03150f (127.18.70.65:45411)
I20260806 02:12:21.588383 18766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:52061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:12:21.610157 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:39021
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:34943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:21.724798 18968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:21.725046 18968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:21.725087 18968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:21.728946 18968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:21.728997 18968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:21.729051 18968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:12:21.734028 18968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:34943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:39021
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:21.734949 18968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:21.735956 18968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:21.745460 18974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:21.745476 18976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:21.745525 18973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:21.745738 18968 server_base.cc:1034] running on GCE node
I20260806 02:12:21.746140 18968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:21.746766 18968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:21.747936 18968 hybrid_clock.cc:648] HybridClock initialized: now 1785982341747904 us; error 40 us; skew 500 ppm
Aug 06 02:12:21 dist-test-slave-6k22 krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982341, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:21.750572 18968 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:12:21.751422 18968 webserver.cc:460] Webserver started at http://127.18.70.66:38159/ using document root <none> and password file <none>
I20260806 02:12:21.752039 18968 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:21.752101 18968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:21.752341 18968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:21.754324 18968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "a269abba905246b69d056248dd7066f1"
format_stamp: "Formatted at 2026-08-06 02:12:21 on dist-test-slave-6k22"
server_key: "c1c59f538d1f1f7e30ccb9b9a0c9f60e"
server_key_iv: "bfcde7835ad4b32f5939f0dcdc6ea823"
server_key_version: "encryptionkey@0"
I20260806 02:12:21.754878 18968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "a269abba905246b69d056248dd7066f1"
format_stamp: "Formatted at 2026-08-06 02:12:21 on dist-test-slave-6k22"
server_key: "c1c59f538d1f1f7e30ccb9b9a0c9f60e"
server_key_iv: "bfcde7835ad4b32f5939f0dcdc6ea823"
server_key_version: "encryptionkey@0"
I20260806 02:12:21.758548 18968 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 02:12:21.761142 18985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:21.762316 18968 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:21.762435 18968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "a269abba905246b69d056248dd7066f1"
format_stamp: "Formatted at 2026-08-06 02:12:21 on dist-test-slave-6k22"
server_key: "c1c59f538d1f1f7e30ccb9b9a0c9f60e"
server_key_iv: "bfcde7835ad4b32f5939f0dcdc6ea823"
server_key_version: "encryptionkey@0"
I20260806 02:12:21.762534 18968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:21.775103 18968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:21.777844 18968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:21.778033 18968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:21.778596 18968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:21.779392 18968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:21.779438 18968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:21.779505 18968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:21.779543 18968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:21.788794 18968 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:36413
I20260806 02:12:21.788821 19102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:36413 every 8 connection(s)
I20260806 02:12:21.789619 18968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:12:21.797086 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 18968
I20260806 02:12:21.797206 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:12:21.797467 18713 external_mini_cluster.cc:1428] Setting key ebefb579a73535541ae693938ae3dc24
Aug 06 02:12:21 dist-test-slave-6k22 krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982341, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:21.803674 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:21.791314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:41481 (local address 127.18.70.126:39021)
0806 02:12:21.791469 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:21.791473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:21.792769 (+ 1296us) server_negotiation.cc:407] Connection header received
0806 02:12:21.793671 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:21.793675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:21.793746 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:21.793850 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:21.794964 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:21.795493 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:21.796625 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:21.796851 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:21.796959 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:21.799780 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:21.799787 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:21.799793 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:21.799827 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:21.801433 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:21.802056 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:21.802058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:21.802059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:21.802143 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:21.802530 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:21.802532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:21.802533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:21.802710 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:21.802791 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:21.803403 (+ 612us) server_negotiation.cc:299] Negotiation successful
0806 02:12:21.803492 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 02:12:21.804853 19105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:21.791687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:39021 (local address 127.18.70.66:41481)
0806 02:12:21.792612 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:21.792653 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:21.793461 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:21.794022 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:21.794029 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:21.794370 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:21.794818 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:21.794831 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:21.795645 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:21.795649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:21.796498 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:21.796506 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:21.797028 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:21.797048 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:21.797268 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:21.797869 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:21.797886 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:21.799548 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:21.801553 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:21.801561 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:21.801579 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:21.801947 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:21.802228 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:21.802233 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:21.802237 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:21.802434 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:21.802826 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:21.802834 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:21.803148 (+ 314us) client_negotiation.cc:769] Sending connection context
0806 02:12:21.803366 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 02:12:21.803736 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":591,"threads_started":1}
I20260806 02:12:21.806018 19103 heartbeater.cc:347] Connected to a master server at 127.18.70.126:39021
I20260806 02:12:21.806272 19103 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:21.806759 19103 heartbeater.cc:511] Master 127.18.70.126:39021 requested a full tablet report, sending...
I20260806 02:12:21.807787 18766 ts_manager.cc:194] Registered new tserver with Master: a269abba905246b69d056248dd7066f1 (127.18.70.66:36413)
I20260806 02:12:21.808336 18766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:41481
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:12:21.827121 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:39021
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:34943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:21.943049 19108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:21.943290 19108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:21.943333 19108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:21.947292 19108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:21.947351 19108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:21.947408 19108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:12:21.952490 19108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:34943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:39021
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:21.953399 19108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:21.954319 19108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:21.964015 19114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:21.964044 19113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:21.964033 19116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:21.964314 19108 server_base.cc:1034] running on GCE node
I20260806 02:12:21.964771 19108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:21.965337 19108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:21.966568 19108 hybrid_clock.cc:648] HybridClock initialized: now 1785982341966555 us; error 30 us; skew 500 ppm
Aug 06 02:12:21 dist-test-slave-6k22 krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982341, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:21.969362 19108 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:12:21.970261 19108 webserver.cc:460] Webserver started at http://127.18.70.67:44635/ using document root <none> and password file <none>
I20260806 02:12:21.970829 19108 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:21.970923 19108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:21.971158 19108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:21.973189 19108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "87cdcf6bacd64d23a29ed3eeefa9b188"
format_stamp: "Formatted at 2026-08-06 02:12:21 on dist-test-slave-6k22"
server_key: "c750ef55d91688bcd0241934e536e532"
server_key_iv: "c077df4dbf9082ec9fd7eab9c47bf8e5"
server_key_version: "encryptionkey@0"
I20260806 02:12:21.973795 19108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "87cdcf6bacd64d23a29ed3eeefa9b188"
format_stamp: "Formatted at 2026-08-06 02:12:21 on dist-test-slave-6k22"
server_key: "c750ef55d91688bcd0241934e536e532"
server_key_iv: "c077df4dbf9082ec9fd7eab9c47bf8e5"
server_key_version: "encryptionkey@0"
I20260806 02:12:21.977465 19108 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 02:12:21.979959 19124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:21.981242 19108 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:21.981376 19108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "87cdcf6bacd64d23a29ed3eeefa9b188"
format_stamp: "Formatted at 2026-08-06 02:12:21 on dist-test-slave-6k22"
server_key: "c750ef55d91688bcd0241934e536e532"
server_key_iv: "c077df4dbf9082ec9fd7eab9c47bf8e5"
server_key_version: "encryptionkey@0"
I20260806 02:12:21.981485 19108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:21.991020 19108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:21.994225 19108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:21.994411 19108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:21.994956 19108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:21.995776 19108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:21.995842 19108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:21.995910 19108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:21.995959 19108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:22.005183 19108 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:46483
I20260806 02:12:22.005223 19241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:46483 every 8 connection(s)
I20260806 02:12:22.006038 19108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:12:22.014379 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 19108
I20260806 02:12:22.014504 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:12:22.014807 18713 external_mini_cluster.cc:1428] Setting key ed7ac57ff33ca296fa0e331ecf1ccf18
Aug 06 02:12:22 dist-test-slave-6k22 krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982341, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:22.019464 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:22.007862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:50055 (local address 127.18.70.126:39021)
0806 02:12:22.008008 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:22.008012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:22.008872 (+ 860us) server_negotiation.cc:407] Connection header received
0806 02:12:22.010052 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:22.010055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:22.010102 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:22.010193 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:22.011360 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:22.012034 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:22.013116 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:22.013345 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:22.013422 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:22.016103 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:22.016107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:22.016109 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:22.016136 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:22.017698 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:22.018188 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:22.018190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:22.018191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:22.018274 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:22.018587 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:22.018589 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:22.018590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:22.018750 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:22.018825 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:22.019259 (+ 434us) server_negotiation.cc:299] Negotiation successful
0806 02:12:22.019351 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 02:12:22.020308 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:22.008265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:39021 (local address 127.18.70.67:50055)
0806 02:12:22.008703 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:22.008747 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:22.009850 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:22.010383 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:22.010389 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:22.010759 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:22.011214 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:22.011226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:22.012142 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:22.012145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:22.013013 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:22.013023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:22.013472 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:22.013474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:22.013610 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:22.014164 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:22.014180 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:22.015941 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:22.017804 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:22.017808 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:22.017818 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:22.018095 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:22.018356 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:22.018369 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:22.018371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:22.018505 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:22.018840 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:22.018845 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:22.019052 (+ 207us) client_negotiation.cc:769] Sending connection context
0806 02:12:22.019202 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 02:12:22.019428 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":113,"threads_started":1}
I20260806 02:12:22.021370 19242 heartbeater.cc:347] Connected to a master server at 127.18.70.126:39021
I20260806 02:12:22.021601 19242 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:22.022071 19242 heartbeater.cc:511] Master 127.18.70.126:39021 requested a full tablet report, sending...
I20260806 02:12:22.023089 18766 ts_manager.cc:194] Registered new tserver with Master: 87cdcf6bacd64d23a29ed3eeefa9b188 (127.18.70.67:46483)
I20260806 02:12:22.023649 18766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:50055
I20260806 02:12:22.029039 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 02:12:22 dist-test-slave-6k22 krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982342, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 02:12:22 dist-test-slave-6k22 krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982342, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.70.65@KRBTEST.COM
I20260806 02:12:22.055124 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:22.046104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35606 (local address 127.18.70.65:45411)
0806 02:12:22.046373 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:22.046388 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:22.046413 (+ 25us) server_negotiation.cc:407] Connection header received
0806 02:12:22.046520 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:22.046525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:22.046705 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:22.046828 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:22.047457 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:22.048236 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:22.049063 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:22.049492 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:22.049600 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:22.051939 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:22.051949 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:22.051961 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:22.051984 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:22.053660 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:22.054040 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:22.054045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:22.054046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:22.054120 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:22.054343 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:22.054346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:22.054347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:22.054557 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:22.054716 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:22.054859 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 02:12:22.054977 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":78,"threads_started":1}
W20260806 02:12:22.056160 18874 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35606
Aug 06 02:12:22 dist-test-slave-6k22 krb5kdc[18720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 02:12:22 dist-test-slave-6k22 krb5kdc[18720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.65@KRBTEST.COM
I20260806 02:12:22.081856 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:22.073941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35610 (local address 127.18.70.65:45411)
0806 02:12:22.074135 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:22.074139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:22.074228 (+ 89us) server_negotiation.cc:407] Connection header received
0806 02:12:22.074334 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:22.074337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:22.074380 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:22.074450 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:22.075109 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:22.075693 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:22.076566 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:22.076791 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:22.076873 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:22.079133 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:22.079136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:22.079138 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:22.079160 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:22.080436 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:22.080790 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:22.080792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:22.080793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:22.080861 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:22.081103 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:22.081105 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:22.081106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:22.081258 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:22.081352 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:22.081559 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 02:12:22.081648 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 02:12:22.082902 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 18830
I20260806 02:12:22.092010 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 18968
I20260806 02:12:22.099892 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 19108
I20260806 02:12:22.106261 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 18733
2026-08-06T02:12:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2317 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:12:22 dist-test-slave-6k22 krb5kdc[19257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:12:22 dist-test-slave-6k22 krb5kdc[19257](info): set up 2 sockets
Aug 06 02:12:22 dist-test-slave-6k22 krb5kdc[19257](info): commencing operation
krb5kdc: starting...
W20260806 02:12:23.177383 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:12:23 dist-test-slave-6k22 krb5kdc[19257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:12:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:12:23.298084 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:36209
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:44605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:36209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:23.409303 19273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:23.409566 19273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:23.409605 19273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:23.413419 19273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:23.413467 19273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:23.413486 19273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:23.413502 19273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:23.413516 19273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:23.418623 19273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:44605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:36209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:36209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:23.419632 19273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:23.420619 19273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:23.428807 19281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:23.428789 19279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:23.428767 19278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:23.428843 19273 server_base.cc:1034] running on GCE node
I20260806 02:12:23.429630 19273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:23.430713 19273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:23.431910 19273 hybrid_clock.cc:648] HybridClock initialized: now 1785982343431870 us; error 51 us; skew 500 ppm
Aug 06 02:12:23 dist-test-slave-6k22 krb5kdc[19257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982343, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:23.434561 19273 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:12:23.435474 19273 webserver.cc:460] Webserver started at http://127.18.70.126:38495/ using document root <none> and password file <none>
I20260806 02:12:23.436048 19273 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:23.436137 19273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:23.436370 19273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:23.438355 19273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "27c18775bb1c4a99a70a20facc2e0e2b"
format_stamp: "Formatted at 2026-08-06 02:12:23 on dist-test-slave-6k22"
server_key: "a655d31bcd92657315e5387c95d900b0"
server_key_iv: "de2c6a58b323fe19a675c098d3b24720"
server_key_version: "encryptionkey@0"
I20260806 02:12:23.438922 19273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "27c18775bb1c4a99a70a20facc2e0e2b"
format_stamp: "Formatted at 2026-08-06 02:12:23 on dist-test-slave-6k22"
server_key: "a655d31bcd92657315e5387c95d900b0"
server_key_iv: "de2c6a58b323fe19a675c098d3b24720"
server_key_version: "encryptionkey@0"
I20260806 02:12:23.442659 19273 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 02:12:23.445140 19289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:23.446394 19273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:23.446537 19273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "27c18775bb1c4a99a70a20facc2e0e2b"
format_stamp: "Formatted at 2026-08-06 02:12:23 on dist-test-slave-6k22"
server_key: "a655d31bcd92657315e5387c95d900b0"
server_key_iv: "de2c6a58b323fe19a675c098d3b24720"
server_key_version: "encryptionkey@0"
I20260806 02:12:23.446652 19273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:23.479305 19273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:23.482407 19273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:23.482615 19273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:23.489971 19273 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:36209
I20260806 02:12:23.489997 19344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:36209 every 8 connection(s)
I20260806 02:12:23.490921 19273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:23.493848 19345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:23.494324 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 19273
I20260806 02:12:23.494441 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:23.494735 18713 external_mini_cluster.cc:1428] Setting key 8c7ff931e7b84f593fcf1256bff32a9a
I20260806 02:12:23.501065 19345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b: Bootstrap starting.
Aug 06 02:12:23 dist-test-slave-6k22 krb5kdc[19257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:23.503959 19345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:23.504890 19345 log.cc:824] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:23.507182 19345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b: No bootstrap required, opened a new log
I20260806 02:12:23.509069 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:23.496102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57158 (local address 127.18.70.126:36209)
0806 02:12:23.496537 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:23.496552 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:23.496587 (+ 35us) server_negotiation.cc:407] Connection header received
0806 02:12:23.497239 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:23.497263 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:23.497574 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:23.497876 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:23.498771 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:23.499810 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:23.500934 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:23.501439 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:23.501557 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:23.504395 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:23.504401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:23.504417 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:23.504447 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:23.506214 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:23.506735 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:23.506740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:23.506744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:23.506848 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:23.507129 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:23.507131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:23.507132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:23.507631 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:23.507788 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:23.508067 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 02:12:23.508373 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":142,"threads_started":1}
I20260806 02:12:23.511622 19345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c18775bb1c4a99a70a20facc2e0e2b" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 36209 } }
I20260806 02:12:23.511871 19345 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:23.511950 19345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c18775bb1c4a99a70a20facc2e0e2b, State: Initialized, Role: FOLLOWER
I20260806 02:12:23.512415 19345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c18775bb1c4a99a70a20facc2e0e2b" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 36209 } }
I20260806 02:12:23.512565 19345 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:23.512678 19345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:23.512779 19345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:23.513643 19345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c18775bb1c4a99a70a20facc2e0e2b" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 36209 } }
I20260806 02:12:23.513993 19345 leader_election.cc:302] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27c18775bb1c4a99a70a20facc2e0e2b; no voters:
I20260806 02:12:23.514292 19345 leader_election.cc:288] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:23.514416 19350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:23.514674 19350 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [term 1 LEADER]: Becoming Leader. State: Replica: 27c18775bb1c4a99a70a20facc2e0e2b, State: Running, Role: LEADER
I20260806 02:12:23.515049 19350 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c18775bb1c4a99a70a20facc2e0e2b" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 36209 } }
I20260806 02:12:23.515687 19345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:23.516825 19352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27c18775bb1c4a99a70a20facc2e0e2b. Latest consensus state: current_term: 1 leader_uuid: "27c18775bb1c4a99a70a20facc2e0e2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c18775bb1c4a99a70a20facc2e0e2b" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 36209 } } }
I20260806 02:12:23.516957 19352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:23.517432 19351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27c18775bb1c4a99a70a20facc2e0e2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c18775bb1c4a99a70a20facc2e0e2b" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 36209 } } }
I20260806 02:12:23.517549 19351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:23.517422 19359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:23.520762 19359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:23.526158 19359 catalog_manager.cc:1347] Generated new cluster ID: dc963ddddd614ec99138b936d0aa5caf
I20260806 02:12:23.526221 19359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:23.542137 19359 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:23.543114 19359 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:23.547879 19359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27c18775bb1c4a99a70a20facc2e0e2b: Generated new TSK 0
I20260806 02:12:23.548501 19359 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:12:23.576118 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:36209
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:44605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:23.691391 19371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:23.691715 19371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:23.691766 19371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:23.695672 19371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:23.695731 19371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:23.695788 19371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:12:23.700865 19371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:44605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:36209
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:23.701895 19371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:23.702901 19371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:23.712437 19377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:23.712437 19376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:23.712440 19379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:23.712685 19371 server_base.cc:1034] running on GCE node
I20260806 02:12:23.713182 19371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:23.713814 19371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:23.714998 19371 hybrid_clock.cc:648] HybridClock initialized: now 1785982343714980 us; error 29 us; skew 500 ppm
Aug 06 02:12:23 dist-test-slave-6k22 krb5kdc[19257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982343, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:23.717941 19371 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:12:23.718815 19371 webserver.cc:460] Webserver started at http://127.18.70.65:34501/ using document root <none> and password file <none>
I20260806 02:12:23.719419 19371 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:23.719513 19371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:23.719780 19371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:23.721827 19371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "0d1edc0c50ea4e49b42f736be65f7d7a"
format_stamp: "Formatted at 2026-08-06 02:12:23 on dist-test-slave-6k22"
server_key: "1b1fc558d9e7ab99821466cbb533f74c"
server_key_iv: "c106fd6d55f8c11adfd497abf28ef080"
server_key_version: "encryptionkey@0"
I20260806 02:12:23.722396 19371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "0d1edc0c50ea4e49b42f736be65f7d7a"
format_stamp: "Formatted at 2026-08-06 02:12:23 on dist-test-slave-6k22"
server_key: "1b1fc558d9e7ab99821466cbb533f74c"
server_key_iv: "c106fd6d55f8c11adfd497abf28ef080"
server_key_version: "encryptionkey@0"
I20260806 02:12:23.726096 19371 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:12:23.728628 19387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:23.729868 19371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:23.730005 19371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "0d1edc0c50ea4e49b42f736be65f7d7a"
format_stamp: "Formatted at 2026-08-06 02:12:23 on dist-test-slave-6k22"
server_key: "1b1fc558d9e7ab99821466cbb533f74c"
server_key_iv: "c106fd6d55f8c11adfd497abf28ef080"
server_key_version: "encryptionkey@0"
I20260806 02:12:23.730116 19371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:23.741178 19371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:23.744007 19371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:23.744212 19371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:23.744796 19371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:23.745609 19371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:23.745682 19371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:23.745745 19371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:23.745783 19371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:23.755457 19371 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:38235
I20260806 02:12:23.755479 19503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:38235 every 8 connection(s)
I20260806 02:12:23.756421 19371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:12:23.762799 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 19371
I20260806 02:12:23.762923 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:12:23.763173 18713 external_mini_cluster.cc:1428] Setting key 3135ef72f3cd81b3a83e4ce19f19dd66
Aug 06 02:12:23 dist-test-slave-6k22 krb5kdc[19257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982343, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:23.770910 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:23.758207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:60157 (local address 127.18.70.126:36209)
0806 02:12:23.758368 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:23.758372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:23.759140 (+ 768us) server_negotiation.cc:407] Connection header received
0806 02:12:23.760881 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:23.760884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:23.760942 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:23.761052 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:23.762221 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:23.762789 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:23.764312 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:23.764614 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:23.764739 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:23.767166 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:23.767170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:23.767173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:23.767196 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:23.768820 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:23.769356 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:23.769360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:23.769362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:23.769446 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:23.769808 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:23.769814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:23.769815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:23.769998 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:23.770108 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:23.770608 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 02:12:23.770730 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 02:12:23.771791 19506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:23.758558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:36209 (local address 127.18.70.65:60157)
0806 02:12:23.758989 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:23.759031 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:23.760677 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:23.761248 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:23.761255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:23.761611 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:23.762073 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:23.762086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:23.762919 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:23.762923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:23.764169 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:23.764179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:23.764792 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:23.764795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:23.764936 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:23.765529 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:23.765545 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:23.766977 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:23.768953 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:23.768959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:23.768974 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:23.769240 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:23.769541 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:23.769546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:23.769548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:23.769693 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:23.770108 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:23.770116 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:23.770371 (+ 255us) client_negotiation.cc:769] Sending connection context
0806 02:12:23.770565 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 02:12:23.770896 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":121,"threads_started":1}
I20260806 02:12:23.773197 19504 heartbeater.cc:347] Connected to a master server at 127.18.70.126:36209
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
I20260806 02:12:23.773454 19504 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:23.773937 19504 heartbeater.cc:511] Master 127.18.70.126:36209 requested a full tablet report, sending...
I20260806 02:12:23.775494 19308 ts_manager.cc:194] Registered new tserver with Master: 0d1edc0c50ea4e49b42f736be65f7d7a (127.18.70.65:38235)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
I20260806 02:12:23.776719 19308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:60157
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:12:23.791963 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:36209
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:44605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:23.916774 19509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:23.917026 19509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:23.917075 19509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:23.921592 19509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:23.921660 19509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:23.921725 19509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:12:23.927143 19509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:44605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:36209
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:23.928184 19509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:23.929121 19509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 02:12:23.938725 19509 server_base.cc:1034] running on GCE node
W20260806 02:12:23.938684 19515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:23.938694 19514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:23.938920 19517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:23.939286 19509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:23.940048 19509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:23.941238 19509 hybrid_clock.cc:648] HybridClock initialized: now 1785982343941221 us; error 33 us; skew 500 ppm
Aug 06 02:12:23 dist-test-slave-6k22 krb5kdc[19257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982343, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:23.944065 19509 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:12:23.944979 19509 webserver.cc:460] Webserver started at http://127.18.70.66:43799/ using document root <none> and password file <none>
I20260806 02:12:23.945530 19509 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:23.945621 19509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:23.945830 19509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:23.947898 19509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "2143bd7fb2ad498698535eeb4c9c5a6c"
format_stamp: "Formatted at 2026-08-06 02:12:23 on dist-test-slave-6k22"
server_key: "0f3055d16a5d30464f7093cdf7da1c50"
server_key_iv: "7aa9b3f9fd4f6a9c0dded165f0efa607"
server_key_version: "encryptionkey@0"
I20260806 02:12:23.948504 19509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "2143bd7fb2ad498698535eeb4c9c5a6c"
format_stamp: "Formatted at 2026-08-06 02:12:23 on dist-test-slave-6k22"
server_key: "0f3055d16a5d30464f7093cdf7da1c50"
server_key_iv: "7aa9b3f9fd4f6a9c0dded165f0efa607"
server_key_version: "encryptionkey@0"
I20260806 02:12:23.952483 19509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 02:12:23.954824 19524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:23.956089 19509 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:12:23.956221 19509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "2143bd7fb2ad498698535eeb4c9c5a6c"
format_stamp: "Formatted at 2026-08-06 02:12:23 on dist-test-slave-6k22"
server_key: "0f3055d16a5d30464f7093cdf7da1c50"
server_key_iv: "7aa9b3f9fd4f6a9c0dded165f0efa607"
server_key_version: "encryptionkey@0"
I20260806 02:12:23.956329 19509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:23.979105 19509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:23.982213 19509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:23.982451 19509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:23.983136 19509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:23.984090 19509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:23.984176 19509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:23.984268 19509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:23.984318 19509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:23.993541 19509 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:45797
I20260806 02:12:23.993573 19640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:45797 every 8 connection(s)
I20260806 02:12:23.994431 19509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:12:23.998869 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 19509
I20260806 02:12:23.999025 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:12:23.999356 18713 external_mini_cluster.cc:1428] Setting key 251a7ffb40771a6c655ab9e7ddf0367a
Aug 06 02:12:24 dist-test-slave-6k22 krb5kdc[19257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982343, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:24.010797 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:23.996236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:41691 (local address 127.18.70.126:36209)
0806 02:12:23.996433 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:23.996437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:23.997302 (+ 865us) server_negotiation.cc:407] Connection header received
0806 02:12:23.998646 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:23.998651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:23.998725 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:23.998855 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:24.000751 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.001446 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.003300 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.003780 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.003930 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.006455 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:24.006462 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:24.006466 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:24.006503 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:24.008293 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:24.009073 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:24.009076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:24.009078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:24.009158 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:24.009621 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:24.009624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:24.009625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:24.009794 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:24.009883 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:24.010583 (+ 700us) server_negotiation.cc:299] Negotiation successful
0806 02:12:24.010677 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
I20260806 02:12:24.012241 19643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:23.996709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:36209 (local address 127.18.70.66:41691)
0806 02:12:23.997149 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:23.997192 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:23.998320 (+ 1128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:23.999040 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:23.999049 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:23.999524 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:24.000122 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.000141 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.001599 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.001605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.003120 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.003137 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.003948 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.003963 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.004104 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.004795 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:24.004812 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:24.006253 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:24.008467 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:24.008477 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:24.008499 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:24.008951 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:24.009271 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:24.009277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:24.009281 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:24.009514 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:24.009918 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:24.009928 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:24.010297 (+ 369us) client_negotiation.cc:769] Sending connection context
0806 02:12:24.010559 (+ 262us) client_negotiation.cc:240] Negotiation successful
0806 02:12:24.010947 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":121,"threads_started":1}
I20260806 02:12:24.013598 19641 heartbeater.cc:347] Connected to a master server at 127.18.70.126:36209
I20260806 02:12:24.013860 19641 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:24.014441 19641 heartbeater.cc:511] Master 127.18.70.126:36209 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
I20260806 02:12:24.015790 19308 ts_manager.cc:194] Registered new tserver with Master: 2143bd7fb2ad498698535eeb4c9c5a6c (127.18.70.66:45797)
I20260806 02:12:24.016676 19308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:41691
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:12:24.032905 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:36209
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:44605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:24.149991 19646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:24.150279 19646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:24.150377 19646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:24.154481 19646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:24.154567 19646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:24.154663 19646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:12:24.159802 19646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:44605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:36209
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:24.160794 19646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:24.161772 19646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:24.171053 19652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:24.171058 19654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:24.171160 19651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:24.171411 19646 server_base.cc:1034] running on GCE node
I20260806 02:12:24.171823 19646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:24.172441 19646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:24.173643 19646 hybrid_clock.cc:648] HybridClock initialized: now 1785982344173609 us; error 51 us; skew 500 ppm
Aug 06 02:12:24 dist-test-slave-6k22 krb5kdc[19257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982344, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:24.177345 19646 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:12:24.178483 19646 webserver.cc:460] Webserver started at http://127.18.70.67:46421/ using document root <none> and password file <none>
I20260806 02:12:24.179078 19646 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:24.179164 19646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:24.179414 19646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:24.181972 19646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "f3216adb51be48d596c2cea80346e999"
format_stamp: "Formatted at 2026-08-06 02:12:24 on dist-test-slave-6k22"
server_key: "e91dde0606fdad637a0392cd8ddb5c7a"
server_key_iv: "6c23612abb6bd2f9e016da728f367b89"
server_key_version: "encryptionkey@0"
I20260806 02:12:24.182614 19646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "f3216adb51be48d596c2cea80346e999"
format_stamp: "Formatted at 2026-08-06 02:12:24 on dist-test-slave-6k22"
server_key: "e91dde0606fdad637a0392cd8ddb5c7a"
server_key_iv: "6c23612abb6bd2f9e016da728f367b89"
server_key_version: "encryptionkey@0"
I20260806 02:12:24.186164 19646 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 02:12:24.188566 19661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:24.189688 19646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:24.189827 19646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "f3216adb51be48d596c2cea80346e999"
format_stamp: "Formatted at 2026-08-06 02:12:24 on dist-test-slave-6k22"
server_key: "e91dde0606fdad637a0392cd8ddb5c7a"
server_key_iv: "6c23612abb6bd2f9e016da728f367b89"
server_key_version: "encryptionkey@0"
I20260806 02:12:24.189930 19646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:24.205250 19646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:24.208384 19646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:24.208580 19646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:24.209128 19646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:24.209950 19646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:24.210021 19646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:24.210088 19646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:24.210141 19646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:24.219393 19646 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:41835
I20260806 02:12:24.219429 19777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:41835 every 8 connection(s)
I20260806 02:12:24.220258 19646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
Aug 06 02:12:24 dist-test-slave-6k22 krb5kdc[19257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982344, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:24.230437 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 19646
I20260806 02:12:24.230583 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:12:24.230964 18713 external_mini_cluster.cc:1428] Setting key c337f42c2cd787495029b8e7a7f17650
I20260806 02:12:24.233912 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.221887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:36129 (local address 127.18.70.126:36209)
0806 02:12:24.222047 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:24.222052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:24.222836 (+ 784us) server_negotiation.cc:407] Connection header received
0806 02:12:24.224051 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:24.224054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:24.224115 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:24.224186 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:24.225784 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.226362 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.227374 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.227663 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.227779 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.230146 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:24.230149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:24.230151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:24.230174 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:24.231863 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:24.232376 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:24.232380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:24.232382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:24.232467 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:24.232812 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:24.232816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:24.232817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:24.232995 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:24.233103 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:24.233595 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 02:12:24.233714 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 02:12:24.234614 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.222243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:36209 (local address 127.18.70.67:36129)
0806 02:12:24.222653 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:24.222692 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:24.223862 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:24.224340 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:24.224345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:24.224922 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:24.225640 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.225655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.226487 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.226489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.227273 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.227282 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.227821 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.227827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.228007 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.228600 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:24.228615 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:24.229996 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:24.231988 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:24.231993 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:24.232007 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:24.232267 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:24.232565 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:24.232569 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:24.232571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:24.232710 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:24.233106 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:24.233115 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:24.233353 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 02:12:24.233550 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 02:12:24.233852 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":110,"threads_started":1}
I20260806 02:12:24.235735 19778 heartbeater.cc:347] Connected to a master server at 127.18.70.126:36209
I20260806 02:12:24.235996 19778 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:24.236485 19778 heartbeater.cc:511] Master 127.18.70.126:36209 requested a full tablet report, sending...
I20260806 02:12:24.237459 19308 ts_manager.cc:194] Registered new tserver with Master: f3216adb51be48d596c2cea80346e999 (127.18.70.67:41835)
I20260806 02:12:24.237990 19308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:36129
I20260806 02:12:24.244920 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:12:24.256456 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.249017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57170 (local address 127.18.70.126:36209)
0806 02:12:24.249129 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:24.249134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:24.249337 (+ 203us) server_negotiation.cc:407] Connection header received
0806 02:12:24.249452 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:24.249454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:24.249489 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:24.249585 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:24.250312 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.250825 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.251737 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.251956 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.252034 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.253398 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:24.253400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:24.253402 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:24.253422 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:24.254806 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:24.255283 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:24.255285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:24.255287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:24.255354 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:24.255681 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:24.255683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:24.255684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:24.255906 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:24.256010 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:24.256266 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 02:12:24.256354 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 02:12:24.262193 19308 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:12:24.265419 19308 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:24.280447 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.274932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:38235 (local address 127.0.0.1:40744)
0806 02:12:24.275237 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:24.275264 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:24.275351 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:24.276089 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:24.276091 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:24.276105 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:24.276198 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.276204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.277270 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.277277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.278687 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.278701 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.279832 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.279835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.279966 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.280006 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 02:12:24.280122 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 02:12:24.280243 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":92,"thread_start_us":95,"threads_started":1}
I20260806 02:12:24.280663 19789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.274205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:41835 (local address 127.0.0.1:57380)
0806 02:12:24.274711 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:24.274766 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:24.274896 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:24.275929 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:24.275934 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:24.275961 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:24.276102 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.276108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.277662 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.277665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.279006 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.279016 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.280135 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.280139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.280227 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.280327 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 02:12:24.280448 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 02:12:24.280521 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":152,"thread_start_us":93,"threads_started":1}
I20260806 02:12:24.280719 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.274378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57380 (local address 127.18.70.67:41835)
0806 02:12:24.275358 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:24.275371 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:24.275396 (+ 25us) server_negotiation.cc:407] Connection header received
0806 02:12:24.275480 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:24.275485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:24.275755 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:24.276258 (+ 503us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:24.276421 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.277525 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.279143 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.279984 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.280129 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.280258 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:24.280484 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 02:12:24.280573 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":871,"thread_start_us":165,"threads_started":1}
I20260806 02:12:24.280810 19793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.274945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58394 (local address 127.18.70.66:45797)
0806 02:12:24.275721 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:24.275734 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:24.275763 (+ 29us) server_negotiation.cc:407] Connection header received
0806 02:12:24.275826 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:24.275830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:24.275970 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:24.276121 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:24.276560 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.277644 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.279076 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.279718 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.279838 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.280042 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:24.280533 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 02:12:24.280646 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":124,"threads_started":1}
I20260806 02:12:24.281205 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.275246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40744 (local address 127.18.70.65:38235)
0806 02:12:24.275673 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:24.275680 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:24.275702 (+ 22us) server_negotiation.cc:407] Connection header received
0806 02:12:24.275782 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:24.275789 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:24.275971 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:24.276122 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:24.276317 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.277129 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.278836 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.279702 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.279820 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.280036 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:24.280964 (+ 928us) server_negotiation.cc:299] Negotiation successful
0806 02:12:24.281067 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":113,"threads_started":1}
I20260806 02:12:24.281342 19790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.274378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:45797 (local address 127.0.0.1:58394)
0806 02:12:24.274986 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:24.275001 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:24.275099 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:24.276301 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:24.276304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:24.276323 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:24.276430 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.276436 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.277789 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.277792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.278953 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.278963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.280110 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.280116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.280227 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.280320 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 02:12:24.280390 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 02:12:24.280448 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":124,"thread_start_us":118,"threads_started":1}
I20260806 02:12:24.283919 19575 tablet_service.cc:1467] Processing CreateTablet for tablet d68fecdeab9a470c9d692d79d81f8d08 (DEFAULT_TABLE table=test-table [id=b326320a0542457f9597f67c399d1387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:24.283952 19438 tablet_service.cc:1467] Processing CreateTablet for tablet d68fecdeab9a470c9d692d79d81f8d08 (DEFAULT_TABLE table=test-table [id=b326320a0542457f9597f67c399d1387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:24.283955 19711 tablet_service.cc:1467] Processing CreateTablet for tablet d68fecdeab9a470c9d692d79d81f8d08 (DEFAULT_TABLE table=test-table [id=b326320a0542457f9597f67c399d1387]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:24.284915 19575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68fecdeab9a470c9d692d79d81f8d08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:24.284957 19711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68fecdeab9a470c9d692d79d81f8d08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:24.284957 19438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68fecdeab9a470c9d692d79d81f8d08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:24.291165 19795 tablet_bootstrap.cc:492] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c: Bootstrap starting.
I20260806 02:12:24.292613 19797 tablet_bootstrap.cc:492] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999: Bootstrap starting.
I20260806 02:12:24.293895 19795 tablet_bootstrap.cc:654] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:24.294528 19797 tablet_bootstrap.cc:654] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:24.294761 19795 log.cc:824] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:24.295499 19796 tablet_bootstrap.cc:492] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a: Bootstrap starting.
I20260806 02:12:24.295786 19797 log.cc:824] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:24.296941 19795 tablet_bootstrap.cc:492] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c: No bootstrap required, opened a new log
I20260806 02:12:24.297137 19795 ts_tablet_manager.cc:1397] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 02:12:24.297596 19797 tablet_bootstrap.cc:492] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999: No bootstrap required, opened a new log
I20260806 02:12:24.297725 19796 tablet_bootstrap.cc:654] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:24.297834 19797 ts_tablet_manager.cc:1397] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 02:12:24.298580 19796 log.cc:824] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:24.300594 19796 tablet_bootstrap.cc:492] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a: No bootstrap required, opened a new log
I20260806 02:12:24.300784 19796 ts_tablet_manager.cc:1397] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 02:12:24.301030 19797 raft_consensus.cc:348] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3216adb51be48d596c2cea80346e999" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41835 } } peers { permanent_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45797 } } peers { permanent_uuid: "0d1edc0c50ea4e49b42f736be65f7d7a" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 38235 } }
I20260806 02:12:24.301054 19795 raft_consensus.cc:348] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3216adb51be48d596c2cea80346e999" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41835 } } peers { permanent_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45797 } } peers { permanent_uuid: "0d1edc0c50ea4e49b42f736be65f7d7a" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 38235 } }
I20260806 02:12:24.301280 19797 raft_consensus.cc:374] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:24.301357 19795 raft_consensus.cc:374] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:24.301367 19797 raft_consensus.cc:729] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3216adb51be48d596c2cea80346e999, State: Initialized, Role: FOLLOWER
I20260806 02:12:24.301424 19795 raft_consensus.cc:729] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2143bd7fb2ad498698535eeb4c9c5a6c, State: Initialized, Role: FOLLOWER
I20260806 02:12:24.301858 19797 consensus_queue.cc:260] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3216adb51be48d596c2cea80346e999" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41835 } } peers { permanent_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45797 } } peers { permanent_uuid: "0d1edc0c50ea4e49b42f736be65f7d7a" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 38235 } }
I20260806 02:12:24.301926 19795 consensus_queue.cc:260] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3216adb51be48d596c2cea80346e999" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41835 } } peers { permanent_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45797 } } peers { permanent_uuid: "0d1edc0c50ea4e49b42f736be65f7d7a" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 38235 } }
I20260806 02:12:24.302863 19797 ts_tablet_manager.cc:1428] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 02:12:24.302963 19795 ts_tablet_manager.cc:1428] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 02:12:24.303267 19778 heartbeater.cc:503] Master 127.18.70.126:36209 was elected leader, sending a full tablet report...
I20260806 02:12:24.303380 19641 heartbeater.cc:503] Master 127.18.70.126:36209 was elected leader, sending a full tablet report...
I20260806 02:12:24.305275 19796 raft_consensus.cc:348] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3216adb51be48d596c2cea80346e999" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41835 } } peers { permanent_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45797 } } peers { permanent_uuid: "0d1edc0c50ea4e49b42f736be65f7d7a" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 38235 } }
I20260806 02:12:24.305538 19796 raft_consensus.cc:374] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:24.305614 19796 raft_consensus.cc:729] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d1edc0c50ea4e49b42f736be65f7d7a, State: Initialized, Role: FOLLOWER
I20260806 02:12:24.306049 19796 consensus_queue.cc:260] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3216adb51be48d596c2cea80346e999" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41835 } } peers { permanent_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45797 } } peers { permanent_uuid: "0d1edc0c50ea4e49b42f736be65f7d7a" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 38235 } }
I20260806 02:12:24.307063 19796 ts_tablet_manager.cc:1428] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 02:12:24.307677 19504 heartbeater.cc:503] Master 127.18.70.126:36209 was elected leader, sending a full tablet report...
I20260806 02:12:24.391598 19802 raft_consensus.cc:482] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:24.391901 19802 raft_consensus.cc:504] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3216adb51be48d596c2cea80346e999" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41835 } } peers { permanent_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45797 } } peers { permanent_uuid: "0d1edc0c50ea4e49b42f736be65f7d7a" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 38235 } }
I20260806 02:12:24.393317 19802 leader_election.cc:288] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3216adb51be48d596c2cea80346e999 (127.18.70.67:41835), 0d1edc0c50ea4e49b42f736be65f7d7a (127.18.70.65:38235)
I20260806 02:12:24.397806 19643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.393590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:41835 (local address 127.18.70.66:38047)
0806 02:12:24.393742 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:24.393757 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:24.393868 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:24.394177 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:24.394180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:24.394205 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:24.394355 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.394362 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.395345 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.395347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.396610 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.396621 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.397421 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.397423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.397501 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.397521 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 02:12:24.397602 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 02:12:24.397669 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 02:12:24.397862 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.393756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:38047 (local address 127.18.70.67:41835)
0806 02:12:24.393911 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:24.393918 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:24.393934 (+ 16us) server_negotiation.cc:407] Connection header received
0806 02:12:24.394005 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:24.394009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:24.394057 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:24.394134 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:24.394503 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.395238 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.396738 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.397306 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.397428 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.397482 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:24.397640 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 02:12:24.397725 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 02:12:24.398036 19804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.393943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:38235 (local address 127.18.70.66:58759)
0806 02:12:24.394389 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:24.394404 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:24.394496 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:24.394725 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:24.394727 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:24.394741 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:24.394838 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.394842 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.395677 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.395680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.396798 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:24.396805 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.397635 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.397638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:24.397727 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.397760 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 02:12:24.397833 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 02:12:24.397899 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":148,"threads_started":1}
I20260806 02:12:24.398070 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.394098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:58759 (local address 127.18.70.65:38235)
0806 02:12:24.394246 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:24.394250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:24.394486 (+ 236us) server_negotiation.cc:407] Connection header received
0806 02:12:24.394577 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:24.394580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:24.394629 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:24.394708 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:24.394939 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.395593 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.396904 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.397521 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.397637 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.397682 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:24.397872 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 02:12:24.397953 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 02:12:24.398806 19458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68fecdeab9a470c9d692d79d81f8d08" candidate_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d1edc0c50ea4e49b42f736be65f7d7a" is_pre_election: true
I20260806 02:12:24.399171 19458 raft_consensus.cc:2454] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2143bd7fb2ad498698535eeb4c9c5a6c in term 0.
I20260806 02:12:24.399608 19526 leader_election.cc:302] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d1edc0c50ea4e49b42f736be65f7d7a, 2143bd7fb2ad498698535eeb4c9c5a6c; no voters:
I20260806 02:12:24.399842 19802 raft_consensus.cc:2781] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:24.399964 19802 raft_consensus.cc:482] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:24.400027 19802 raft_consensus.cc:3037] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:24.401137 19802 raft_consensus.cc:504] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3216adb51be48d596c2cea80346e999" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41835 } } peers { permanent_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45797 } } peers { permanent_uuid: "0d1edc0c50ea4e49b42f736be65f7d7a" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 38235 } }
I20260806 02:12:24.401368 19731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68fecdeab9a470c9d692d79d81f8d08" candidate_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3216adb51be48d596c2cea80346e999" is_pre_election: true
I20260806 02:12:24.401486 19802 leader_election.cc:288] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [CANDIDATE]: Term 1 election: Requested vote from peers f3216adb51be48d596c2cea80346e999 (127.18.70.67:41835), 0d1edc0c50ea4e49b42f736be65f7d7a (127.18.70.65:38235)
I20260806 02:12:24.401702 19731 raft_consensus.cc:2454] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2143bd7fb2ad498698535eeb4c9c5a6c in term 0.
I20260806 02:12:24.401885 19458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68fecdeab9a470c9d692d79d81f8d08" candidate_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d1edc0c50ea4e49b42f736be65f7d7a"
I20260806 02:12:24.401953 19730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68fecdeab9a470c9d692d79d81f8d08" candidate_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3216adb51be48d596c2cea80346e999"
I20260806 02:12:24.402005 19458 raft_consensus.cc:3037] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:24.402061 19730 raft_consensus.cc:3037] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:24.403334 19458 raft_consensus.cc:2454] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2143bd7fb2ad498698535eeb4c9c5a6c in term 1.
I20260806 02:12:24.403334 19730 raft_consensus.cc:2454] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2143bd7fb2ad498698535eeb4c9c5a6c in term 1.
I20260806 02:12:24.403690 19526 leader_election.cc:302] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2143bd7fb2ad498698535eeb4c9c5a6c, f3216adb51be48d596c2cea80346e999; no voters:
I20260806 02:12:24.403855 19802 raft_consensus.cc:2781] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:24.404091 19802 raft_consensus.cc:686] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [term 1 LEADER]: Becoming Leader. State: Replica: 2143bd7fb2ad498698535eeb4c9c5a6c, State: Running, Role: LEADER
I20260806 02:12:24.404443 19802 consensus_queue.cc:237] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3216adb51be48d596c2cea80346e999" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41835 } } peers { permanent_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45797 } } peers { permanent_uuid: "0d1edc0c50ea4e49b42f736be65f7d7a" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 38235 } }
I20260806 02:12:24.407887 19307 catalog_manager.cc:5495] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2143bd7fb2ad498698535eeb4c9c5a6c (127.18.70.66). New cstate: current_term: 1 leader_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3216adb51be48d596c2cea80346e999" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2143bd7fb2ad498698535eeb4c9c5a6c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d1edc0c50ea4e49b42f736be65f7d7a" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 38235 } health_report { overall_health: UNKNOWN } } }
Aug 06 02:12:24 dist-test-slave-6k22 krb5kdc[19257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.65@KRBTEST.COM
I20260806 02:12:24.435791 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.424208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.18.70.65:38235)
0806 02:12:24.424373 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:24.424378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:24.424555 (+ 177us) server_negotiation.cc:407] Connection header received
0806 02:12:24.424733 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:24.424738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:24.424809 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:24.424904 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:24.425855 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.426755 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.428640 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.429055 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.429186 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.432079 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:24.432094 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:24.432112 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:24.432143 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:24.433938 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:24.434389 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:24.434396 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:24.434398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:24.434510 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:24.434809 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:24.434813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:24.434815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:24.435079 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:24.435315 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:24.435470 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 02:12:24.435622 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 02:12:24 dist-test-slave-6k22 krb5kdc[19257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982344, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260806 02:12:24.471184 19779 tablet.cc:2369] T d68fecdeab9a470c9d692d79d81f8d08 P f3216adb51be48d596c2cea80346e999: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 02:12:24 dist-test-slave-6k22 krb5kdc[19257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982344, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.70.65@KRBTEST.COM
I20260806 02:12:24.477094 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.462955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40770 (local address 127.18.70.65:38235)
0806 02:12:24.467722 (+ 4767us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:24.467726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:24.467740 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:12:24.467786 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:24.467788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:24.467829 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:24.467903 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:24.468708 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.469255 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.470128 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.470363 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.470472 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.473761 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:24.473765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:24.473766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:24.473791 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:24.475250 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:24.475705 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:24.475711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:24.475717 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:24.475800 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:24.476140 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:24.476144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:24.476145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:24.476318 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:24.476458 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:24.476797 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 02:12:24.476925 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4655}
Aug 06 02:12:24 dist-test-slave-6k22 krb5kdc[19257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260806 02:12:24.495741 19642 tablet.cc:2369] T d68fecdeab9a470c9d692d79d81f8d08 P 2143bd7fb2ad498698535eeb4c9c5a6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 02:12:24 dist-test-slave-6k22 krb5kdc[19257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.65@KRBTEST.COM
I20260806 02:12:24.504374 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:24.495354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40776 (local address 127.18.70.65:38235)
0806 02:12:24.495486 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:24.495491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:24.495716 (+ 225us) server_negotiation.cc:407] Connection header received
0806 02:12:24.495920 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:24.495923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:24.495977 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:24.496022 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:24.496761 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.497562 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.498388 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:24.498734 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:24.498833 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:24.501334 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:24.501337 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:24.501340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:24.501365 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:24.502838 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:24.503233 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:24.503236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:24.503238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:24.503324 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:24.503588 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:24.503591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:24.503593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:24.503777 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:24.503899 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:24.504136 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 02:12:24.504263 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 02:12:24.505801 19418 tablet_service.cc:2908] Rejecting scan request for tablet d68fecdeab9a470c9d692d79d81f8d08: Uninitialized: clean time has not yet been initialized
I20260806 02:12:24.507762 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 19371
W20260806 02:12:24.507823 19505 tablet.cc:2369] T d68fecdeab9a470c9d692d79d81f8d08 P 0d1edc0c50ea4e49b42f736be65f7d7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:12:24.515883 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 19509
I20260806 02:12:24.523464 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 19646
I20260806 02:12:24.530404 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 19273
2026-08-06T02:12:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2424 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:12:24 dist-test-slave-6k22 krb5kdc[19819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:12:24 dist-test-slave-6k22 krb5kdc[19819](info): set up 2 sockets
Aug 06 02:12:24 dist-test-slave-6k22 krb5kdc[19819](info): commencing operation
krb5kdc: starting...
W20260806 02:12:25.592994 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:12:25 dist-test-slave-6k22 krb5kdc[19819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:12:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:12:25.716388 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:39215
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:39215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:25.829030 19833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:25.829330 19833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:25.829404 19833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:25.833416 19833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:25.833500 19833 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 02:12:25.833534 19833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:25.833575 19833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:25.833609 19833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:25.833645 19833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:25.838894 19833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:39215
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:39215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:25.839946 19833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:25.840981 19833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:25.849581 19839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:25.849609 19838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:25.849773 19833 server_base.cc:1034] running on GCE node
W20260806 02:12:25.849594 19841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:25.850395 19833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:25.851522 19833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:25.852730 19833 hybrid_clock.cc:648] HybridClock initialized: now 1785982345852701 us; error 40 us; skew 500 ppm
Aug 06 02:12:25 dist-test-slave-6k22 krb5kdc[19819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982345, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:25.861815 19833 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:12:25.862753 19833 webserver.cc:460] Webserver started at http://127.18.70.126:37065/ using document root <none> and password file <none>
I20260806 02:12:25.863307 19833 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:25.863368 19833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:25.863538 19833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:25.865638 19833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "b55f35e6a23f4822a7082cec72302832"
format_stamp: "Formatted at 2026-08-06 02:12:25 on dist-test-slave-6k22"
server_key: "7288840257f2258e51aec3e630668792"
server_key_iv: "4dda411bf83544057f757f3729359fc7"
server_key_version: "encryptionkey@0"
I20260806 02:12:25.866173 19833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "b55f35e6a23f4822a7082cec72302832"
format_stamp: "Formatted at 2026-08-06 02:12:25 on dist-test-slave-6k22"
server_key: "7288840257f2258e51aec3e630668792"
server_key_iv: "4dda411bf83544057f757f3729359fc7"
server_key_version: "encryptionkey@0"
I20260806 02:12:25.869882 19833 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:12:25.872236 19849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:25.873521 19833 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:12:25.873670 19833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "b55f35e6a23f4822a7082cec72302832"
format_stamp: "Formatted at 2026-08-06 02:12:25 on dist-test-slave-6k22"
server_key: "7288840257f2258e51aec3e630668792"
server_key_iv: "4dda411bf83544057f757f3729359fc7"
server_key_version: "encryptionkey@0"
I20260806 02:12:25.873796 19833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:25.887967 19833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:25.891165 19833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:25.891369 19833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:25.899921 19833 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:39215
I20260806 02:12:25.899942 19912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:39215 every 8 connection(s)
I20260806 02:12:25.900884 19833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:25.902199 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 19833
I20260806 02:12:25.902299 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:25.902566 18713 external_mini_cluster.cc:1428] Setting key 58a2ae287dd80fa47b84e9cc1a4cadb8
I20260806 02:12:25.903844 19913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:25.909757 19913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832: Bootstrap starting.
Aug 06 02:12:25 dist-test-slave-6k22 krb5kdc[19819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:25.912232 19913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:25.913470 19913 log.cc:824] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:25.915859 19913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832: No bootstrap required, opened a new log
I20260806 02:12:25.916800 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:25.903983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.18.70.126:39215)
0806 02:12:25.904405 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:25.904420 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:25.904456 (+ 36us) server_negotiation.cc:407] Connection header received
0806 02:12:25.905082 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:25.905098 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:25.905411 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:25.905760 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:25.906550 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:25.907545 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:25.908499 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:25.908971 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:25.909161 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:25.911683 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:25.911688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:25.911701 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:25.911735 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:25.913517 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:25.914004 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:25.914010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:25.914014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:25.914140 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:25.914468 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:25.914472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:25.914473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:25.915040 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:25.915248 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:25.915487 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 02:12:25.915827 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":146,"threads_started":1}
I20260806 02:12:25.919543 19913 raft_consensus.cc:348] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55f35e6a23f4822a7082cec72302832" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39215 } }
I20260806 02:12:25.919858 19913 raft_consensus.cc:374] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:25.919919 19913 raft_consensus.cc:729] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b55f35e6a23f4822a7082cec72302832, State: Initialized, Role: FOLLOWER
I20260806 02:12:25.920289 19913 consensus_queue.cc:260] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55f35e6a23f4822a7082cec72302832" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39215 } }
I20260806 02:12:25.920405 19913 raft_consensus.cc:388] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:25.920454 19913 raft_consensus.cc:482] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:25.920531 19913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:25.921478 19913 raft_consensus.cc:504] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55f35e6a23f4822a7082cec72302832" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39215 } }
I20260806 02:12:25.921777 19913 leader_election.cc:302] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b55f35e6a23f4822a7082cec72302832; no voters:
I20260806 02:12:25.922034 19913 leader_election.cc:288] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:25.922185 19918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:25.922428 19918 raft_consensus.cc:686] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [term 1 LEADER]: Becoming Leader. State: Replica: b55f35e6a23f4822a7082cec72302832, State: Running, Role: LEADER
I20260806 02:12:25.922766 19918 consensus_queue.cc:237] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55f35e6a23f4822a7082cec72302832" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39215 } }
I20260806 02:12:25.923228 19913 sys_catalog.cc:564] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:25.924484 19919 sys_catalog.cc:455] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b55f35e6a23f4822a7082cec72302832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55f35e6a23f4822a7082cec72302832" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39215 } } }
I20260806 02:12:25.924615 19919 sys_catalog.cc:458] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:25.925722 19920 sys_catalog.cc:455] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b55f35e6a23f4822a7082cec72302832. Latest consensus state: current_term: 1 leader_uuid: "b55f35e6a23f4822a7082cec72302832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55f35e6a23f4822a7082cec72302832" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39215 } } }
I20260806 02:12:25.925904 19920 sys_catalog.cc:458] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832 [sys.catalog]: This master's current role is: LEADER
W20260806 02:12:25.928292 19934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:12:25.928357 19934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:12:25.928436 19927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:25.929128 19927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:25.935017 19927 catalog_manager.cc:1347] Generated new cluster ID: cf154874351c4c728a77776a0f38579d
I20260806 02:12:25.935089 19927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:25.953819 19927 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:25.954803 19927 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:25.962826 19927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832: Generated new TSK 0
I20260806 02:12:25.963531 19927 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:12:25.999110 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:39215
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:26.124425 19940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:26.124773 19940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:26.124862 19940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:26.129033 19940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:26.129127 19940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:26.129196 19940 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 02:12:26.129284 19940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:12:26.134449 19940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.70.126:39215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:26.135479 19940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:26.136530 19940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:26.146247 19949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:26.146209 19946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:26.146477 19940 server_base.cc:1034] running on GCE node
W20260806 02:12:26.146209 19947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:26.146934 19940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:26.147608 19940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:26.148793 19940 hybrid_clock.cc:648] HybridClock initialized: now 1785982346148777 us; error 27 us; skew 500 ppm
Aug 06 02:12:26 dist-test-slave-6k22 krb5kdc[19819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982346, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:26.151481 19940 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:12:26.152384 19940 webserver.cc:460] Webserver started at http://127.18.70.65:37171/ using document root <none> and password file <none>
I20260806 02:12:26.152907 19940 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:26.152966 19940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:26.153131 19940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:26.155123 19940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "b033bb2eea6244ca942d3e31a6fd2b29"
format_stamp: "Formatted at 2026-08-06 02:12:26 on dist-test-slave-6k22"
server_key: "33f2efcbad6a738572420c57c1539ef5"
server_key_iv: "ff2d5ce7b6a1fae0c371286f9272a0db"
server_key_version: "encryptionkey@0"
I20260806 02:12:26.155715 19940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "b033bb2eea6244ca942d3e31a6fd2b29"
format_stamp: "Formatted at 2026-08-06 02:12:26 on dist-test-slave-6k22"
server_key: "33f2efcbad6a738572420c57c1539ef5"
server_key_iv: "ff2d5ce7b6a1fae0c371286f9272a0db"
server_key_version: "encryptionkey@0"
I20260806 02:12:26.159242 19940 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:12:26.161676 19956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:26.163203 19940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 02:12:26.163331 19940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "b033bb2eea6244ca942d3e31a6fd2b29"
format_stamp: "Formatted at 2026-08-06 02:12:26 on dist-test-slave-6k22"
server_key: "33f2efcbad6a738572420c57c1539ef5"
server_key_iv: "ff2d5ce7b6a1fae0c371286f9272a0db"
server_key_version: "encryptionkey@0"
I20260806 02:12:26.163430 19940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:26.175464 19940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:26.179654 19940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:26.179889 19940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:26.181780 19940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:26.181842 19940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:26.181905 19940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:26.181934 19940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:26.191762 19940 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:39993
I20260806 02:12:26.191861 20075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:39993 every 8 connection(s)
Aug 06 02:12:26 dist-test-slave-6k22 krb5kdc[19819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982346, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:26.192863 19940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:12:26.196276 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.182987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:49989 (local address 127.18.70.126:39215)
0806 02:12:26.183136 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:26.183141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:26.183895 (+ 754us) server_negotiation.cc:407] Connection header received
0806 02:12:26.185127 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:26.185131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:26.185189 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:26.185290 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:26.186946 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.187893 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.189353 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.189763 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.189902 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.192771 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:26.192774 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:26.192776 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:26.192801 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:26.194491 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.194966 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.194968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.194970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.195045 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.195333 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.195335 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.195336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.195496 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:26.195617 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:26.196064 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 02:12:26.196162 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 02:12:26.196579 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 19940
I20260806 02:12:26.196686 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:12:26.196965 18713 external_mini_cluster.cc:1428] Setting key 19d8c5e1874059af5868267deb79b4df
I20260806 02:12:26.197237 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.183217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:39215 (local address 127.18.70.65:49989)
0806 02:12:26.183726 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:26.183779 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:26.184866 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:26.185497 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:26.185509 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:26.186026 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:26.186759 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.186782 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.188036 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.188042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.189212 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.189224 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.189937 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.189941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.190159 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.190830 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:26.190855 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:26.192604 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:26.194603 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:26.194608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:26.194617 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:26.194880 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:26.195122 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:26.195124 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:26.195125 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:26.195255 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:26.195635 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:26.195641 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:26.195867 (+ 226us) client_negotiation.cc:769] Sending connection context
0806 02:12:26.196015 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 02:12:26.196236 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":140,"threads_started":1}
I20260806 02:12:26.198506 20076 heartbeater.cc:347] Connected to a master server at 127.18.70.126:39215
I20260806 02:12:26.198673 20076 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:26.199210 20076 heartbeater.cc:511] Master 127.18.70.126:39215 requested a full tablet report, sending...
I20260806 02:12:26.200937 19865 ts_manager.cc:194] Registered new tserver with Master: b033bb2eea6244ca942d3e31a6fd2b29 (127.18.70.65:39993)
I20260806 02:12:26.202555 19865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:49989
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
I20260806 02:12:26.208345 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.200890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:59317 (local address 127.18.70.126:39215)
0806 02:12:26.201005 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:26.201011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:26.201412 (+ 401us) server_negotiation.cc:407] Connection header received
0806 02:12:26.201541 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:26.201544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:26.201582 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:26.201666 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:26.202094 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.202587 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.203645 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.203957 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.204037 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.205295 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:26.205298 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:26.205301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:26.205324 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:26.206654 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.207073 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.207077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.207078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.207161 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.207442 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.207446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.207447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.207693 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:26.207803 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:26.208053 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 02:12:26.208174 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 02:12:26.208367 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.201033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:39215 (local address 127.18.70.65:59317)
0806 02:12:26.201319 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:26.201340 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:26.201447 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:26.201681 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:26.201684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:26.201889 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:26.201982 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.201987 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.202826 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.202828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.203487 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.203498 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.204180 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.204183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.204269 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.204768 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:26.204782 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:26.205160 (+ 378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:26.206775 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:26.206778 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:26.206780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:26.206974 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:26.207261 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:26.207264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:26.207266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:26.207352 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:26.207794 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:26.207797 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:26.207892 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 02:12:26.208000 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 02:12:26.208174 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":132,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:12:26.224822 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:39215
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:26.337661 20086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:26.337899 20086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:26.337966 20086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:26.342020 20086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:26.342072 20086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:26.342142 20086 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 02:12:26.342236 20086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:12:26.347301 20086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.70.126:39215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:26.348299 20086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:26.349321 20086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:26.358850 20094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:26.358973 20092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:26.358973 20091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:26.359270 20086 server_base.cc:1034] running on GCE node
I20260806 02:12:26.359732 20086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:26.360421 20086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:26.361608 20086 hybrid_clock.cc:648] HybridClock initialized: now 1785982346361593 us; error 26 us; skew 500 ppm
Aug 06 02:12:26 dist-test-slave-6k22 krb5kdc[19819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982346, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:26.364303 20086 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:12:26.365159 20086 webserver.cc:460] Webserver started at http://127.18.70.66:42519/ using document root <none> and password file <none>
I20260806 02:12:26.365731 20086 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:26.365824 20086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:26.366091 20086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:26.368335 20086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "4e228b1005184465982a2a11ee1f51c3"
format_stamp: "Formatted at 2026-08-06 02:12:26 on dist-test-slave-6k22"
server_key: "916f511dfbaf5cfb544a20148c8f85e2"
server_key_iv: "0d235ee4abbd24fb62f149d93b00df2c"
server_key_version: "encryptionkey@0"
I20260806 02:12:26.368916 20086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "4e228b1005184465982a2a11ee1f51c3"
format_stamp: "Formatted at 2026-08-06 02:12:26 on dist-test-slave-6k22"
server_key: "916f511dfbaf5cfb544a20148c8f85e2"
server_key_iv: "0d235ee4abbd24fb62f149d93b00df2c"
server_key_version: "encryptionkey@0"
I20260806 02:12:26.372916 20086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 02:12:26.375730 20101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:26.377068 20086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 02:12:26.377355 20086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "4e228b1005184465982a2a11ee1f51c3"
format_stamp: "Formatted at 2026-08-06 02:12:26 on dist-test-slave-6k22"
server_key: "916f511dfbaf5cfb544a20148c8f85e2"
server_key_iv: "0d235ee4abbd24fb62f149d93b00df2c"
server_key_version: "encryptionkey@0"
I20260806 02:12:26.377482 20086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:26.396740 20086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:26.400193 20086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:26.400411 20086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:26.402021 20086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:26.402099 20086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:26.402172 20086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:26.402225 20086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 02:12:26 dist-test-slave-6k22 krb5kdc[19819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982346, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:26.414942 20086 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:33111
I20260806 02:12:26.414991 20219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:33111 every 8 connection(s)
I20260806 02:12:26.416206 20086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:12:26.416941 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.402940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:37955 (local address 127.18.70.126:39215)
0806 02:12:26.403093 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:26.403097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:26.404277 (+ 1180us) server_negotiation.cc:407] Connection header received
0806 02:12:26.405370 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:26.405374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:26.405439 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:26.405504 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:26.407197 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.408207 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.409508 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.409800 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.409922 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.412707 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:26.412711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:26.412714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:26.412750 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:26.414560 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.415173 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.415177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.415186 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.415283 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.415658 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.415661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.415663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.415926 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:26.416041 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:26.416601 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 02:12:26.416755 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:12:26.417701 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.403602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:39215 (local address 127.18.70.66:37955)
0806 02:12:26.404117 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:26.404159 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:26.405085 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:26.405993 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:26.406003 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:26.406525 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:26.407015 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.407028 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.408356 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.408360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.409374 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.409385 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.409976 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.409980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.410166 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.410824 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:26.410845 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:26.412541 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:26.414694 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:26.414701 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:26.414716 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:26.415075 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:26.415386 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:26.415388 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:26.415390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:26.415520 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:26.416045 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:26.416050 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:26.416314 (+ 264us) client_negotiation.cc:769] Sending connection context
0806 02:12:26.416482 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 02:12:26.416800 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":150,"threads_started":1}
I20260806 02:12:26.418949 20220 heartbeater.cc:347] Connected to a master server at 127.18.70.126:39215
I20260806 02:12:26.419122 20220 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:26.419754 20220 heartbeater.cc:511] Master 127.18.70.126:39215 requested a full tablet report, sending...
I20260806 02:12:26.420835 19864 ts_manager.cc:194] Registered new tserver with Master: 4e228b1005184465982a2a11ee1f51c3 (127.18.70.66:33111)
I20260806 02:12:26.421156 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 20086
I20260806 02:12:26.421311 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:12:26.421633 18713 external_mini_cluster.cc:1428] Setting key bb457b37d18576d17e600a3ea6a5afc8
I20260806 02:12:26.421731 19864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:37955
I20260806 02:12:26.429857 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.421094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:39215 (local address 127.18.70.66:50517)
0806 02:12:26.421527 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:26.421541 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:26.421618 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:26.422419 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:26.422422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:26.422611 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:26.422698 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.422704 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.423802 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.423808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.424772 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.424786 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.425274 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.425281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.425390 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.425920 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:26.425935 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:26.426239 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:26.428302 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:26.428304 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:26.428306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:26.428487 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:26.428830 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:26.428832 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:26.428833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:26.428903 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:26.429460 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:26.429462 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:26.429536 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 02:12:26.429613 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 02:12:26.429711 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":117,"threads_started":1}
I20260806 02:12:26.430078 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.422083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:50517 (local address 127.18.70.126:39215)
0806 02:12:26.422216 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:26.422221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:26.422234 (+ 13us) server_negotiation.cc:407] Connection header received
0806 02:12:26.422279 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:26.422282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:26.422324 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:26.422402 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:26.422863 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.423619 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.424892 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.425110 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.425274 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.426377 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:26.426382 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:26.426386 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:26.426419 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:26.428189 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.428590 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.428598 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.428603 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.428717 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.429009 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.429015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.429020 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.429338 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:26.429457 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:26.429719 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 02:12:26.429850 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:12:26.454010 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:39215
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:26.565953 20230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:26.566241 20230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:26.566327 20230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:26.570447 20230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:26.570535 20230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:26.570609 20230 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 02:12:26.570691 20230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:12:26.576323 20230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.70.126:39215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:26.577549 20230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:26.578739 20230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:26.588510 20235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:26.588575 20238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:26.588698 20230 server_base.cc:1034] running on GCE node
W20260806 02:12:26.588516 20236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:26.589200 20230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:26.589885 20230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:26.591086 20230 hybrid_clock.cc:648] HybridClock initialized: now 1785982346591055 us; error 40 us; skew 500 ppm
Aug 06 02:12:26 dist-test-slave-6k22 krb5kdc[19819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982346, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:26.593986 20230 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:12:26.594907 20230 webserver.cc:460] Webserver started at http://127.18.70.67:43739/ using document root <none> and password file <none>
I20260806 02:12:26.595484 20230 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:26.595600 20230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:26.595855 20230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:26.597914 20230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "c357ceebcd904ba88f8c1d2bc70100da"
format_stamp: "Formatted at 2026-08-06 02:12:26 on dist-test-slave-6k22"
server_key: "293a5a1f9dedc7cc4a5dce118f3e704e"
server_key_iv: "8b13eb152b0b98df30440661c56be65d"
server_key_version: "encryptionkey@0"
I20260806 02:12:26.598558 20230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "c357ceebcd904ba88f8c1d2bc70100da"
format_stamp: "Formatted at 2026-08-06 02:12:26 on dist-test-slave-6k22"
server_key: "293a5a1f9dedc7cc4a5dce118f3e704e"
server_key_iv: "8b13eb152b0b98df30440661c56be65d"
server_key_version: "encryptionkey@0"
I20260806 02:12:26.602396 20230 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 02:12:26.604918 20245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:26.606101 20230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:26.606238 20230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "c357ceebcd904ba88f8c1d2bc70100da"
format_stamp: "Formatted at 2026-08-06 02:12:26 on dist-test-slave-6k22"
server_key: "293a5a1f9dedc7cc4a5dce118f3e704e"
server_key_iv: "8b13eb152b0b98df30440661c56be65d"
server_key_version: "encryptionkey@0"
I20260806 02:12:26.606348 20230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:26.624087 20230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:26.627375 20230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:26.627640 20230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:26.629153 20230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:26.629226 20230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:26.629293 20230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:26.629349 20230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 02:12:26 dist-test-slave-6k22 krb5kdc[19819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982346, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:26.640574 20230 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:42595
I20260806 02:12:26.641687 20230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:12:26.642652 20362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:42595 every 8 connection(s)
I20260806 02:12:26.644654 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.629886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:33293 (local address 127.18.70.126:39215)
0806 02:12:26.630020 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:26.630025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:26.631760 (+ 1735us) server_negotiation.cc:407] Connection header received
0806 02:12:26.633014 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:26.633018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:26.633082 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:26.633168 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:26.634854 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.635779 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.637611 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.637825 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.637912 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.641124 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:26.641128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:26.641130 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:26.641161 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:26.642824 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.643275 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.643278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.643279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.643355 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.643664 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.643674 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.643675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.643858 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:26.643940 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:26.644434 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 02:12:26.644531 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:12:26.645500 20254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.631001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:39215 (local address 127.18.70.67:33293)
0806 02:12:26.631601 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:26.631657 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:26.632765 (+ 1108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:26.633369 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:26.633382 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:26.633894 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:26.634650 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.634672 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.635943 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.635948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.637477 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.637490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.637990 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.637996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.638243 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.638943 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:26.638967 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:26.640947 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:26.642921 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:26.642926 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:26.642937 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:26.643185 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:26.643419 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:26.643421 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:26.643423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:26.643540 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:26.643983 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:26.643989 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:26.644214 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 02:12:26.644371 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 02:12:26.644611 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":166,"threads_started":1}
I20260806 02:12:26.646808 20363 heartbeater.cc:347] Connected to a master server at 127.18.70.126:39215
I20260806 02:12:26.646956 20363 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:26.647547 20363 heartbeater.cc:511] Master 127.18.70.126:39215 requested a full tablet report, sending...
I20260806 02:12:26.648682 19865 ts_manager.cc:194] Registered new tserver with Master: c357ceebcd904ba88f8c1d2bc70100da (127.18.70.67:42595)
I20260806 02:12:26.649415 19865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:33293
I20260806 02:12:26.650163 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 20230
I20260806 02:12:26.650296 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:12:26.650612 18713 external_mini_cluster.cc:1428] Setting key 03107035b7c7ede66077e43ba5145a64
I20260806 02:12:26.652704 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:12:26.656767 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.649199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:49579 (local address 127.18.70.126:39215)
0806 02:12:26.649334 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:26.649338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:26.649447 (+ 109us) server_negotiation.cc:407] Connection header received
0806 02:12:26.649616 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:26.649618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:26.649680 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:26.649760 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:26.650242 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.650846 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.651946 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.652252 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.652360 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.653487 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:26.653490 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:26.653493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:26.653517 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:26.655094 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.655510 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.655514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.655516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.655670 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.655956 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.655960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.655961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.656137 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:26.656252 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:26.656500 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 02:12:26.656611 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 02:12:26.656785 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.649071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:39215 (local address 127.18.70.67:49579)
0806 02:12:26.649374 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:26.649389 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:26.649494 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:26.649790 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:26.649793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:26.650006 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:26.650121 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.650127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.650971 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.650974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.651817 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.651829 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.652361 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.652364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.652456 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.653010 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:26.653024 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:26.653350 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:26.655203 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:26.655208 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:26.655210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:26.655407 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:26.655766 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:26.655770 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:26.655772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:26.655863 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:26.656235 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:26.656238 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:26.656331 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 02:12:26.656444 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 02:12:26.656586 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":103,"threads_started":1}
Aug 06 02:12:26 dist-test-slave-6k22 krb5kdc[19819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982346, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 02:12:26 dist-test-slave-6k22 krb5kdc[19819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982346, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:26.680131 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.672209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35834 (local address 127.18.70.126:39215)
0806 02:12:26.672349 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:26.672352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:26.672364 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:12:26.672403 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:26.672405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:26.672442 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:26.672533 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:26.673374 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.673886 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.674756 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.674966 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.675072 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.677367 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:26.677370 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:26.677372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:26.677394 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:26.678718 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.679092 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.679094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.679096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.679167 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:26.679398 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:26.679400 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:26.679401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:26.679615 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:26.679745 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:26.679921 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 02:12:26.680011 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 02:12:26.683182 19865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:12:26.686316 19865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:26.701188 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.694955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:39993 (local address 127.0.0.1:60096)
0806 02:12:26.695751 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:26.695764 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:26.695866 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:26.696571 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:26.696577 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:26.696599 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:26.696804 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.696813 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.698458 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.698462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.699801 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.699816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.700576 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.700579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.700685 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.700721 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 02:12:26.700831 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 02:12:26.701004 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":107,"thread_start_us":98,"threads_started":1}
I20260806 02:12:26.701233 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.695579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:42595 (local address 127.0.0.1:44302)
0806 02:12:26.695901 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:26.695915 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:26.695989 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:26.696759 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:26.696762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:26.696794 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:26.697093 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.697101 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.698525 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.698528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.699788 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.699799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.700820 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.700823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.700913 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.700935 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 02:12:26.700989 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 02:12:26.701067 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":238,"spinlock_wait_cycles":5120,"thread_start_us":99,"threads_started":1}
I20260806 02:12:26.701236 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.694785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:33111 (local address 127.0.0.1:50222)
0806 02:12:26.695290 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:26.695339 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:26.695475 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:26.696759 (+ 1284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:26.696762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:26.696793 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:26.696987 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.696993 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.698113 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.698117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.699657 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.699668 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.700823 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.700828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.700956 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.700979 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 02:12:26.701052 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 02:12:26.701116 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":133,"spinlock_wait_cycles":4864,"thread_start_us":84,"threads_started":1}
I20260806 02:12:26.701337 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.695514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50222 (local address 127.18.70.66:33111)
0806 02:12:26.696247 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:26.696260 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:26.696284 (+ 24us) server_negotiation.cc:407] Connection header received
0806 02:12:26.696366 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:26.696371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:26.696561 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:26.696883 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:26.697095 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.697996 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.699804 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.700662 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.700820 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.700987 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:26.701080 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 02:12:26.701176 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":126,"threads_started":1}
I20260806 02:12:26.701912 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.695917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44302 (local address 127.18.70.67:42595)
0806 02:12:26.696292 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:26.696304 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:26.696323 (+ 19us) server_negotiation.cc:407] Connection header received
0806 02:12:26.696385 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:26.696389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:26.696561 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:26.696930 (+ 369us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:26.697230 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.698397 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.699932 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.700662 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.701435 (+ 773us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.701606 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:26.701675 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 02:12:26.701777 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":124,"threads_started":1}
I20260806 02:12:26.702178 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.695115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60096 (local address 127.18.70.65:39993)
0806 02:12:26.695482 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:26.695708 (+ 226us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:26.695859 (+ 151us) server_negotiation.cc:407] Connection header received
0806 02:12:26.695981 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:26.695987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:26.696371 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:26.696997 (+ 626us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:26.697146 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.698326 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.699916 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.700449 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.700576 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.700725 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:26.701918 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0806 02:12:26.702048 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":69,"threads_started":1}
I20260806 02:12:26.704795 20296 tablet_service.cc:1467] Processing CreateTablet for tablet 7c15a7326a0d4a04892b0c0fcc73990b (DEFAULT_TABLE table=test-table [id=24971e4e75924a8287b49ecd92e8d2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:26.704881 20152 tablet_service.cc:1467] Processing CreateTablet for tablet 7c15a7326a0d4a04892b0c0fcc73990b (DEFAULT_TABLE table=test-table [id=24971e4e75924a8287b49ecd92e8d2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:26.704874 20006 tablet_service.cc:1467] Processing CreateTablet for tablet 7c15a7326a0d4a04892b0c0fcc73990b (DEFAULT_TABLE table=test-table [id=24971e4e75924a8287b49ecd92e8d2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:26.705693 20296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c15a7326a0d4a04892b0c0fcc73990b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:26.705840 20006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c15a7326a0d4a04892b0c0fcc73990b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:26.705840 20152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c15a7326a0d4a04892b0c0fcc73990b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:26.712952 20387 tablet_bootstrap.cc:492] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da: Bootstrap starting.
I20260806 02:12:26.713593 20388 tablet_bootstrap.cc:492] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29: Bootstrap starting.
I20260806 02:12:26.714125 20389 tablet_bootstrap.cc:492] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3: Bootstrap starting.
I20260806 02:12:26.715732 20387 tablet_bootstrap.cc:654] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:26.715983 20389 tablet_bootstrap.cc:654] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:26.716529 20387 log.cc:824] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:26.716837 20389 log.cc:824] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:26.717648 20388 tablet_bootstrap.cc:654] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:26.718719 20387 tablet_bootstrap.cc:492] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da: No bootstrap required, opened a new log
I20260806 02:12:26.718928 20387 ts_tablet_manager.cc:1397] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 02:12:26.718905 20388 log.cc:824] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:26.719102 20389 tablet_bootstrap.cc:492] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3: No bootstrap required, opened a new log
I20260806 02:12:26.719328 20389 ts_tablet_manager.cc:1397] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 02:12:26.720991 20388 tablet_bootstrap.cc:492] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29: No bootstrap required, opened a new log
I20260806 02:12:26.721176 20388 ts_tablet_manager.cc:1397] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260806 02:12:26.722250 20387 raft_consensus.cc:348] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.722492 20387 raft_consensus.cc:374] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:26.722556 20387 raft_consensus.cc:729] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c357ceebcd904ba88f8c1d2bc70100da, State: Initialized, Role: FOLLOWER
I20260806 02:12:26.723076 20387 consensus_queue.cc:260] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.723356 20389 raft_consensus.cc:348] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.723634 20389 raft_consensus.cc:374] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:26.723870 20389 raft_consensus.cc:729] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e228b1005184465982a2a11ee1f51c3, State: Initialized, Role: FOLLOWER
I20260806 02:12:26.723923 20363 heartbeater.cc:503] Master 127.18.70.126:39215 was elected leader, sending a full tablet report...
I20260806 02:12:26.724447 20389 consensus_queue.cc:260] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.725123 20387 ts_tablet_manager.cc:1428] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260806 02:12:26.725209 20388 raft_consensus.cc:348] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.725472 20389 ts_tablet_manager.cc:1428] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260806 02:12:26.725512 20388 raft_consensus.cc:374] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:26.725565 20388 raft_consensus.cc:729] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b033bb2eea6244ca942d3e31a6fd2b29, State: Initialized, Role: FOLLOWER
I20260806 02:12:26.725807 20220 heartbeater.cc:503] Master 127.18.70.126:39215 was elected leader, sending a full tablet report...
I20260806 02:12:26.725981 20388 consensus_queue.cc:260] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.726533 20076 heartbeater.cc:503] Master 127.18.70.126:39215 was elected leader, sending a full tablet report...
I20260806 02:12:26.726835 20388 ts_tablet_manager.cc:1428] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 02:12:26.892827 20364 tablet.cc:2369] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:12:26.908749 20393 raft_consensus.cc:482] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:26.908977 20393 raft_consensus.cc:504] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.910297 20393 leader_election.cc:288] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e228b1005184465982a2a11ee1f51c3 (127.18.70.66:33111), b033bb2eea6244ca942d3e31a6fd2b29 (127.18.70.65:39993)
I20260806 02:12:26.914373 20254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.910575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:33111 (local address 127.18.70.67:41227)
0806 02:12:26.910833 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:26.910848 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:26.910960 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:26.911347 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:26.911350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:26.911397 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:26.911509 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.911515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.912372 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.912379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.913210 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.913228 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.913903 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.913907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.914023 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.914063 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 02:12:26.914153 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 02:12:26.914241 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":29,"spinlock_wait_cycles":39680}
I20260806 02:12:26.914371 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.910720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:41227 (local address 127.18.70.66:33111)
0806 02:12:26.910863 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:26.910867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:26.910922 (+ 55us) server_negotiation.cc:407] Connection header received
0806 02:12:26.911088 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:26.911091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:26.911140 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:26.911223 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:26.911666 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.912255 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.913356 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.913778 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.913913 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.913970 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:26.914173 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 02:12:26.914250 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 02:12:26.915033 20172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c15a7326a0d4a04892b0c0fcc73990b" candidate_uuid: "c357ceebcd904ba88f8c1d2bc70100da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e228b1005184465982a2a11ee1f51c3" is_pre_election: true
I20260806 02:12:26.915329 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.910575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:39993 (local address 127.18.70.67:56441)
0806 02:12:26.911254 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:26.911265 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:26.911413 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:26.911772 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:26.911775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:26.911792 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:26.912449 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.912460 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.913239 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.913244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.914067 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:26.914076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.914901 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.914905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:26.915012 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.915029 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 02:12:26.915117 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 02:12:26.915181 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":389,"spinlock_wait_cycles":6400,"thread_start_us":162,"threads_started":1}
I20260806 02:12:26.915316 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.910757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:56441 (local address 127.18.70.65:39993)
0806 02:12:26.910878 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:26.910885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:26.911351 (+ 466us) server_negotiation.cc:407] Connection header received
0806 02:12:26.911522 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:26.911525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:26.911658 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:26.911737 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:26.912590 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.913119 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.914203 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.914773 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.914905 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.914951 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:26.915137 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 02:12:26.915208 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:12:26.915341 20172 raft_consensus.cc:2454] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c357ceebcd904ba88f8c1d2bc70100da in term 0.
I20260806 02:12:26.915892 20247 leader_election.cc:302] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e228b1005184465982a2a11ee1f51c3, c357ceebcd904ba88f8c1d2bc70100da; no voters:
I20260806 02:12:26.916132 20028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c15a7326a0d4a04892b0c0fcc73990b" candidate_uuid: "c357ceebcd904ba88f8c1d2bc70100da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" is_pre_election: true
I20260806 02:12:26.916208 20393 raft_consensus.cc:2781] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:26.916296 20393 raft_consensus.cc:482] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:26.916352 20393 raft_consensus.cc:3037] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:26.916453 20028 raft_consensus.cc:2454] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c357ceebcd904ba88f8c1d2bc70100da in term 0.
I20260806 02:12:26.917573 20393 raft_consensus.cc:504] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
W20260806 02:12:26.917768 20221 tablet.cc:2369] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:12:26.917953 20393 leader_election.cc:288] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [CANDIDATE]: Term 1 election: Requested vote from peers 4e228b1005184465982a2a11ee1f51c3 (127.18.70.66:33111), b033bb2eea6244ca942d3e31a6fd2b29 (127.18.70.65:39993)
I20260806 02:12:26.918296 20172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c15a7326a0d4a04892b0c0fcc73990b" candidate_uuid: "c357ceebcd904ba88f8c1d2bc70100da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e228b1005184465982a2a11ee1f51c3"
I20260806 02:12:26.918350 20028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c15a7326a0d4a04892b0c0fcc73990b" candidate_uuid: "c357ceebcd904ba88f8c1d2bc70100da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b033bb2eea6244ca942d3e31a6fd2b29"
I20260806 02:12:26.918468 20172 raft_consensus.cc:3037] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:26.918478 20028 raft_consensus.cc:3037] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:26.919502 20172 raft_consensus.cc:2454] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c357ceebcd904ba88f8c1d2bc70100da in term 1.
I20260806 02:12:26.919663 20028 raft_consensus.cc:2454] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c357ceebcd904ba88f8c1d2bc70100da in term 1.
I20260806 02:12:26.919852 20247 leader_election.cc:302] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e228b1005184465982a2a11ee1f51c3, c357ceebcd904ba88f8c1d2bc70100da; no voters:
I20260806 02:12:26.920049 20393 raft_consensus.cc:2781] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:26.920250 20393 raft_consensus.cc:686] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [term 1 LEADER]: Becoming Leader. State: Replica: c357ceebcd904ba88f8c1d2bc70100da, State: Running, Role: LEADER
I20260806 02:12:26.920564 20393 consensus_queue.cc:237] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.923527 19865 catalog_manager.cc:5495] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da reported cstate change: term changed from 0 to 1, leader changed from <none> to c357ceebcd904ba88f8c1d2bc70100da (127.18.70.67). New cstate: current_term: 1 leader_uuid: "c357ceebcd904ba88f8c1d2bc70100da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } health_report { overall_health: HEALTHY } } }
I20260806 02:12:26.942641 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:26.938401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44314 (local address 127.18.70.67:42595)
0806 02:12:26.938542 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:26.938549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:26.938585 (+ 36us) server_negotiation.cc:407] Connection header received
0806 02:12:26.938835 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:26.938845 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:26.938900 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:26.939007 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:26.939301 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.940056 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.941055 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:26.941271 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:26.941387 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:26.941751 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:26.942183 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:26.942323 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:26.942411 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 02:12:26.942470 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 02:12:26.944623 20077 tablet.cc:2369] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:12:26.949263 20172 raft_consensus.cc:1261] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3 [term 1 FOLLOWER]: Refusing update from remote peer c357ceebcd904ba88f8c1d2bc70100da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:26.950119 20397 consensus_queue.cc:1035] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:26.951524 20028 raft_consensus.cc:1261] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29 [term 1 FOLLOWER]: Refusing update from remote peer c357ceebcd904ba88f8c1d2bc70100da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:26.952193 20397 consensus_queue.cc:1035] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [LEADER]: Connected to new peer: Peer: permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:26.958837 20401 mvcc.cc:205] Tried to move back new op lower bound from 7315383693095825408 to 7315383692988919808. Current Snapshot: MvccSnapshot[applied={T|T < 7315383693095825408}]
I20260806 02:12:26.960052 20404 mvcc.cc:205] Tried to move back new op lower bound from 7315383693095825408 to 7315383692988919808. Current Snapshot: MvccSnapshot[applied={T|T < 7315383693095825408}]
I20260806 02:12:26.972085 19865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:26.972370 19865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:26.975214 19865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832: Sending DeleteTablet for 3 replicas of tablet 7c15a7326a0d4a04892b0c0fcc73990b
I20260806 02:12:26.976085 20152 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c15a7326a0d4a04892b0c0fcc73990b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:26 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:50222
I20260806 02:12:26.976092 20006 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c15a7326a0d4a04892b0c0fcc73990b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:26 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:60096
I20260806 02:12:26.976300 20296 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c15a7326a0d4a04892b0c0fcc73990b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:26 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:44302
I20260806 02:12:26.976716 20412 tablet_replica.cc:330] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da: stopping tablet replica
I20260806 02:12:26.977011 20412 raft_consensus.cc:2229] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:12:26.977339 20412 raft_consensus.cc:2258] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:26.977617 19865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:12:26.978004 19865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:26.978837 20411 tablet_replica.cc:330] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29: stopping tablet replica
I20260806 02:12:26.979156 20411 raft_consensus.cc:2229] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:26.979318 20410 tablet_replica.cc:330] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3: stopping tablet replica
I20260806 02:12:26.979455 20411 raft_consensus.cc:2258] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:26.979861 20412 ts_tablet_manager.cc:1905] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:26.980829 20411 ts_tablet_manager.cc:1905] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:26.983309 20412 ts_tablet_manager.cc:1918] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:26.983424 20412 log.cc:1196] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/7c15a7326a0d4a04892b0c0fcc73990b
I20260806 02:12:26.983511 20411 ts_tablet_manager.cc:1918] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:26.983623 20411 log.cc:1196] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/wal/wals/7c15a7326a0d4a04892b0c0fcc73990b
I20260806 02:12:26.983839 20412 ts_tablet_manager.cc:1939] T 7c15a7326a0d4a04892b0c0fcc73990b P c357ceebcd904ba88f8c1d2bc70100da: Deleting consensus metadata
I20260806 02:12:26.983932 20411 ts_tablet_manager.cc:1939] T 7c15a7326a0d4a04892b0c0fcc73990b P b033bb2eea6244ca942d3e31a6fd2b29: Deleting consensus metadata
I20260806 02:12:26.985069 20410 raft_consensus.cc:2229] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:26.985461 20410 raft_consensus.cc:2258] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:26.985780 20152 tablet_service.cc:1467] Processing CreateTablet for tablet 569bd5aaa2b84c319842f89d6b08f751 (DEFAULT_TABLE table=test-table [id=6b3a954a79094aba965edb0a2cdfe570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:26.985913 20296 tablet_service.cc:1467] Processing CreateTablet for tablet 569bd5aaa2b84c319842f89d6b08f751 (DEFAULT_TABLE table=test-table [id=6b3a954a79094aba965edb0a2cdfe570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:26.986060 20152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 569bd5aaa2b84c319842f89d6b08f751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:26.986613 19850 catalog_manager.cc:4841] TS b033bb2eea6244ca942d3e31a6fd2b29 (127.18.70.65:39993): tablet 7c15a7326a0d4a04892b0c0fcc73990b (table test-table [id=24971e4e75924a8287b49ecd92e8d2ac]) successfully deleted
I20260806 02:12:26.987452 20296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 569bd5aaa2b84c319842f89d6b08f751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:26.987741 20410 ts_tablet_manager.cc:1905] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:26.988497 20389 tablet_bootstrap.cc:492] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3: Bootstrap starting.
I20260806 02:12:26.989411 20389 tablet_bootstrap.cc:654] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:26.990607 20410 ts_tablet_manager.cc:1918] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:26.990688 20006 tablet_service.cc:1467] Processing CreateTablet for tablet 569bd5aaa2b84c319842f89d6b08f751 (DEFAULT_TABLE table=test-table [id=6b3a954a79094aba965edb0a2cdfe570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:26.990846 20410 log.cc:1196] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/7c15a7326a0d4a04892b0c0fcc73990b
I20260806 02:12:26.990942 20006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 569bd5aaa2b84c319842f89d6b08f751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:26.991144 20387 tablet_bootstrap.cc:492] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da: Bootstrap starting.
I20260806 02:12:26.991375 20410 ts_tablet_manager.cc:1939] T 7c15a7326a0d4a04892b0c0fcc73990b P 4e228b1005184465982a2a11ee1f51c3: Deleting consensus metadata
I20260806 02:12:26.992277 20387 tablet_bootstrap.cc:654] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:26.993069 20388 tablet_bootstrap.cc:492] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29: Bootstrap starting.
I20260806 02:12:26.993427 19851 catalog_manager.cc:4841] TS 4e228b1005184465982a2a11ee1f51c3 (127.18.70.66:33111): tablet 7c15a7326a0d4a04892b0c0fcc73990b (table test-table [id=24971e4e75924a8287b49ecd92e8d2ac]) successfully deleted
I20260806 02:12:26.993767 20387 tablet_bootstrap.cc:492] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da: No bootstrap required, opened a new log
I20260806 02:12:26.993855 20387 ts_tablet_manager.cc:1397] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:12:26.994102 20388 tablet_bootstrap.cc:654] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:26.994346 20389 tablet_bootstrap.cc:492] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3: No bootstrap required, opened a new log
I20260806 02:12:26.994434 20389 ts_tablet_manager.cc:1397] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260806 02:12:26.994760 20387 raft_consensus.cc:348] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.994902 20387 raft_consensus.cc:374] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:26.994870 20389 raft_consensus.cc:348] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.994947 20387 raft_consensus.cc:729] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c357ceebcd904ba88f8c1d2bc70100da, State: Initialized, Role: FOLLOWER
I20260806 02:12:26.994969 20389 raft_consensus.cc:374] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:26.994999 20389 raft_consensus.cc:729] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e228b1005184465982a2a11ee1f51c3, State: Initialized, Role: FOLLOWER
I20260806 02:12:26.995117 20389 consensus_queue.cc:260] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.995136 20387 consensus_queue.cc:260] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.995360 20389 ts_tablet_manager.cc:1428] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:26.995433 20387 ts_tablet_manager.cc:1428] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:26.995525 20388 tablet_bootstrap.cc:492] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29: No bootstrap required, opened a new log
I20260806 02:12:26.995644 20388 ts_tablet_manager.cc:1397] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:12:26.996423 20388 raft_consensus.cc:348] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.996464 19853 catalog_manager.cc:4841] TS c357ceebcd904ba88f8c1d2bc70100da (127.18.70.67:42595): tablet 7c15a7326a0d4a04892b0c0fcc73990b (table test-table [id=24971e4e75924a8287b49ecd92e8d2ac]) successfully deleted
I20260806 02:12:26.996541 20388 raft_consensus.cc:374] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:26.996575 20388 raft_consensus.cc:729] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b033bb2eea6244ca942d3e31a6fd2b29, State: Initialized, Role: FOLLOWER
I20260806 02:12:26.996760 20388 consensus_queue.cc:260] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:26.997788 20388 ts_tablet_manager.cc:1428] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:27.046131 20395 raft_consensus.cc:482] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:27.046294 20395 raft_consensus.cc:504] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:27.047607 20395 leader_election.cc:288] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e228b1005184465982a2a11ee1f51c3 (127.18.70.66:33111), c357ceebcd904ba88f8c1d2bc70100da (127.18.70.67:42595)
I20260806 02:12:27.052103 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.048133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:46355 (local address 127.18.70.66:33111)
0806 02:12:27.048270 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:27.048276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:27.048292 (+ 16us) server_negotiation.cc:407] Connection header received
0806 02:12:27.048483 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:27.048489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:27.048566 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:27.048619 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:27.049164 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.049981 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.051071 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.051484 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.051660 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.051717 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:27.051908 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 02:12:27.051981 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 02:12:27.052136 20416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.047896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:33111 (local address 127.18.70.65:46355)
0806 02:12:27.048221 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:27.048237 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:27.048343 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:27.048802 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:27.048807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:27.048826 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:27.049029 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.049034 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.050096 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.050099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.050935 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.050944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.051647 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.051651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.051779 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.051799 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 02:12:27.051890 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 02:12:27.051964 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":38,"spinlock_wait_cycles":9472,"thread_start_us":87,"threads_started":1}
I20260806 02:12:27.052657 20172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "569bd5aaa2b84c319842f89d6b08f751" candidate_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e228b1005184465982a2a11ee1f51c3" is_pre_election: true
I20260806 02:12:27.052825 20172 raft_consensus.cc:2454] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b033bb2eea6244ca942d3e31a6fd2b29 in term 0.
I20260806 02:12:27.053206 19958 leader_election.cc:302] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e228b1005184465982a2a11ee1f51c3, b033bb2eea6244ca942d3e31a6fd2b29; no voters:
I20260806 02:12:27.053507 20395 raft_consensus.cc:2781] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:27.053589 20395 raft_consensus.cc:482] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:27.053642 20395 raft_consensus.cc:3037] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:27.053910 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.048318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:49369 (local address 127.18.70.67:42595)
0806 02:12:27.048443 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:27.048447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:27.048580 (+ 133us) server_negotiation.cc:407] Connection header received
0806 02:12:27.048666 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:27.048669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:27.048724 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:27.048808 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:27.050162 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.050960 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.052579 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.053190 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.053281 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.053330 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:27.053717 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 02:12:27.053792 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 02:12:27.054315 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.047918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:42595 (local address 127.18.70.65:49369)
0806 02:12:27.048462 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:27.048475 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:27.048577 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:27.048824 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:27.048826 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:27.048838 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:27.049016 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.049024 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.051102 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.051106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.052432 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.052444 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.053424 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.053427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.053541 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.053562 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 02:12:27.054096 (+ 534us) client_negotiation.cc:240] Negotiation successful
0806 02:12:27.054181 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":22,"thread_start_us":119,"threads_started":1}
I20260806 02:12:27.054798 20316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "569bd5aaa2b84c319842f89d6b08f751" candidate_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c357ceebcd904ba88f8c1d2bc70100da" is_pre_election: true
I20260806 02:12:27.054955 20395 raft_consensus.cc:504] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:27.055032 20316 raft_consensus.cc:2454] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b033bb2eea6244ca942d3e31a6fd2b29 in term 0.
I20260806 02:12:27.055367 20395 leader_election.cc:288] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [CANDIDATE]: Term 1 election: Requested vote from peers 4e228b1005184465982a2a11ee1f51c3 (127.18.70.66:33111), c357ceebcd904ba88f8c1d2bc70100da (127.18.70.67:42595)
I20260806 02:12:27.055668 20172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "569bd5aaa2b84c319842f89d6b08f751" candidate_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e228b1005184465982a2a11ee1f51c3"
I20260806 02:12:27.055776 20172 raft_consensus.cc:3037] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:27.056921 20172 raft_consensus.cc:2454] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b033bb2eea6244ca942d3e31a6fd2b29 in term 1.
I20260806 02:12:27.057168 20316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "569bd5aaa2b84c319842f89d6b08f751" candidate_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c357ceebcd904ba88f8c1d2bc70100da"
I20260806 02:12:27.057211 19958 leader_election.cc:302] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e228b1005184465982a2a11ee1f51c3, b033bb2eea6244ca942d3e31a6fd2b29; no voters:
I20260806 02:12:27.057282 20316 raft_consensus.cc:3037] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:27.057390 20395 raft_consensus.cc:2781] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:27.057636 20395 raft_consensus.cc:686] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [term 1 LEADER]: Becoming Leader. State: Replica: b033bb2eea6244ca942d3e31a6fd2b29, State: Running, Role: LEADER
I20260806 02:12:27.057924 20395 consensus_queue.cc:237] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:27.058398 20316 raft_consensus.cc:2454] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b033bb2eea6244ca942d3e31a6fd2b29 in term 1.
I20260806 02:12:27.060539 19864 catalog_manager.cc:5495] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 reported cstate change: term changed from 0 to 1, leader changed from <none> to b033bb2eea6244ca942d3e31a6fd2b29 (127.18.70.65). New cstate: current_term: 1 leader_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } health_report { overall_health: UNKNOWN } } }
Aug 06 02:12:27 dist-test-slave-6k22 krb5kdc[19819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982345, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:27.095806 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.085461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:39215 (local address 127.0.0.1:35836)
0806 02:12:27.085798 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:27.085814 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:27.085942 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:27.086245 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:27.086247 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:27.086653 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:27.086762 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.086768 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.087905 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.087926 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.088583 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.088593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.089281 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.089284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.089350 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.090002 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:27.090016 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:27.091849 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:27.093895 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:27.093904 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:27.093909 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:27.094173 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:27.094567 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:27.094570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:27.094572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:27.094660 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:27.095187 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:27.095193 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:27.095299 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 02:12:27.095396 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 02:12:27.095515 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":126,"threads_started":1}
I20260806 02:12:27.095851 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.085535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35836 (local address 127.18.70.126:39215)
0806 02:12:27.085689 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:27.085693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:27.085927 (+ 234us) server_negotiation.cc:407] Connection header received
0806 02:12:27.086039 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:27.086044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:27.086096 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:27.086208 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:27.086895 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.087767 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.088780 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.089102 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.089257 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.092006 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:27.092010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:27.092013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:27.092038 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:27.093767 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:27.094289 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:27.094293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:27.094295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:27.094390 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:27.094823 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:27.094827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:27.094828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:27.095062 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:27.095187 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:27.095494 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 02:12:27.095647 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 02:12:27.100544 19864 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:35836:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 02:12:27.101387 19864 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:27.106463 20006 tablet_service.cc:1467] Processing CreateTablet for tablet ee47f7c30ef448498ae9f151a52f87ed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=379ad7e7d02548679774aaaf90169973]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 02:12:27.106680 20296 tablet_service.cc:1467] Processing CreateTablet for tablet ee47f7c30ef448498ae9f151a52f87ed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=379ad7e7d02548679774aaaf90169973]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 02:12:27.106786 20152 tablet_service.cc:1467] Processing CreateTablet for tablet ee47f7c30ef448498ae9f151a52f87ed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=379ad7e7d02548679774aaaf90169973]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 02:12:27.106981 20006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee47f7c30ef448498ae9f151a52f87ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:27.107016 20296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee47f7c30ef448498ae9f151a52f87ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:27.107210 20152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee47f7c30ef448498ae9f151a52f87ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:27.110674 20389 tablet_bootstrap.cc:492] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3: Bootstrap starting.
I20260806 02:12:27.111392 20388 tablet_bootstrap.cc:492] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29: Bootstrap starting.
I20260806 02:12:27.111750 20389 tablet_bootstrap.cc:654] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:27.112296 20388 tablet_bootstrap.cc:654] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:27.113185 20389 tablet_bootstrap.cc:492] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3: No bootstrap required, opened a new log
I20260806 02:12:27.113269 20389 ts_tablet_manager.cc:1397] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:12:27.113724 20389 raft_consensus.cc:348] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:27.113857 20389 raft_consensus.cc:374] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:27.113898 20389 raft_consensus.cc:729] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e228b1005184465982a2a11ee1f51c3, State: Initialized, Role: FOLLOWER
I20260806 02:12:27.113896 20388 tablet_bootstrap.cc:492] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29: No bootstrap required, opened a new log
I20260806 02:12:27.113989 20388 ts_tablet_manager.cc:1397] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:12:27.114217 20389 consensus_queue.cc:260] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:27.114491 20394 tablet_replica.cc:439] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } } }
I20260806 02:12:27.114620 20394 tablet_replica.cc:442] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:27.114535 20388 raft_consensus.cc:348] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:27.114665 20388 raft_consensus.cc:374] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:27.114708 20388 raft_consensus.cc:729] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b033bb2eea6244ca942d3e31a6fd2b29, State: Initialized, Role: FOLLOWER
I20260806 02:12:27.115307 20389 ts_tablet_manager.cc:1428] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 02:12:27.115458 20388 consensus_queue.cc:260] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:27.115761 20387 tablet_bootstrap.cc:492] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da: Bootstrap starting.
I20260806 02:12:27.116886 20387 tablet_bootstrap.cc:654] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:27.118548 20387 tablet_bootstrap.cc:492] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da: No bootstrap required, opened a new log
I20260806 02:12:27.118631 20387 ts_tablet_manager.cc:1397] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:12:27.119011 20387 raft_consensus.cc:348] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:27.119158 20387 raft_consensus.cc:374] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:27.119207 20387 raft_consensus.cc:729] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c357ceebcd904ba88f8c1d2bc70100da, State: Initialized, Role: FOLLOWER
I20260806 02:12:27.119535 20387 consensus_queue.cc:260] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:27.119848 20397 tablet_replica.cc:439] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } } }
I20260806 02:12:27.119993 20397 tablet_replica.cc:442] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:27.120369 20387 ts_tablet_manager.cc:1428] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:27.120720 20388 ts_tablet_manager.cc:1428] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260806 02:12:27.120658 20395 tablet_replica.cc:439] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } } }
I20260806 02:12:27.120805 20395 tablet_replica.cc:442] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:27.185596 20394 raft_consensus.cc:482] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:27.185763 20394 raft_consensus.cc:504] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:27.186915 20394 leader_election.cc:288] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b033bb2eea6244ca942d3e31a6fd2b29 (127.18.70.65:39993), c357ceebcd904ba88f8c1d2bc70100da (127.18.70.67:42595)
I20260806 02:12:27.192333 20429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.187133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:39993 (local address 127.18.70.66:44007)
0806 02:12:27.187527 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:27.187539 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:27.187711 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:27.187980 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:27.187983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:27.187999 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:27.188232 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.188244 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.189300 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.189304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.190913 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.190924 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.191845 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.191848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.191985 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.192013 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 02:12:27.192095 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 02:12:27.192166 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":174,"thread_start_us":152,"threads_started":1}
I20260806 02:12:27.192349 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.187319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:44007 (local address 127.18.70.65:39993)
0806 02:12:27.187452 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:27.187458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:27.187661 (+ 203us) server_negotiation.cc:407] Connection header received
0806 02:12:27.187829 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:27.187837 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:27.187880 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:27.187966 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:27.188349 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.189161 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.191058 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.191717 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.191840 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.191895 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:27.192130 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 02:12:27.192230 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:12:27.192425 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.187335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:55757 (local address 127.18.70.67:42595)
0806 02:12:27.187548 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:27.187552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:27.187754 (+ 202us) server_negotiation.cc:407] Connection header received
0806 02:12:27.187913 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:27.187916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:27.187964 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:27.188062 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:27.188391 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.189169 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.191058 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.191734 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.191844 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.191895 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:27.192225 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 02:12:27.192305 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 02:12:27.192854 20028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c30ef448498ae9f151a52f87ed" candidate_uuid: "4e228b1005184465982a2a11ee1f51c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" is_pre_election: true
I20260806 02:12:27.193001 20028 raft_consensus.cc:2454] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e228b1005184465982a2a11ee1f51c3 in term 0.
I20260806 02:12:27.193038 20430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.187166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:42595 (local address 127.18.70.66:55757)
0806 02:12:27.187661 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:27.187673 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:27.187794 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:27.188072 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:27.188075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:27.188089 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:27.188289 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.188298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.189300 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.189304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.190913 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.190924 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.191952 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.191961 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.192048 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.192089 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 02:12:27.192794 (+ 705us) client_negotiation.cc:240] Negotiation successful
0806 02:12:27.192881 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":217,"thread_start_us":103,"threads_started":1}
I20260806 02:12:27.193368 20102 leader_election.cc:302] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e228b1005184465982a2a11ee1f51c3, b033bb2eea6244ca942d3e31a6fd2b29; no voters:
I20260806 02:12:27.193473 20316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c30ef448498ae9f151a52f87ed" candidate_uuid: "4e228b1005184465982a2a11ee1f51c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c357ceebcd904ba88f8c1d2bc70100da" is_pre_election: true
I20260806 02:12:27.193634 20394 raft_consensus.cc:2781] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:27.193625 20316 raft_consensus.cc:2454] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e228b1005184465982a2a11ee1f51c3 in term 0.
I20260806 02:12:27.193687 20394 raft_consensus.cc:482] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:27.193724 20394 raft_consensus.cc:3037] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:27.194787 20394 raft_consensus.cc:504] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:27.195104 20394 leader_election.cc:288] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [CANDIDATE]: Term 1 election: Requested vote from peers b033bb2eea6244ca942d3e31a6fd2b29 (127.18.70.65:39993), c357ceebcd904ba88f8c1d2bc70100da (127.18.70.67:42595)
I20260806 02:12:27.195369 20028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c30ef448498ae9f151a52f87ed" candidate_uuid: "4e228b1005184465982a2a11ee1f51c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b033bb2eea6244ca942d3e31a6fd2b29"
I20260806 02:12:27.195428 20316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c30ef448498ae9f151a52f87ed" candidate_uuid: "4e228b1005184465982a2a11ee1f51c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c357ceebcd904ba88f8c1d2bc70100da"
I20260806 02:12:27.195472 20028 raft_consensus.cc:3037] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:27.195533 20316 raft_consensus.cc:3037] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:27.196420 20316 raft_consensus.cc:2454] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e228b1005184465982a2a11ee1f51c3 in term 1.
I20260806 02:12:27.196542 20028 raft_consensus.cc:2454] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e228b1005184465982a2a11ee1f51c3 in term 1.
I20260806 02:12:27.196779 20105 leader_election.cc:302] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e228b1005184465982a2a11ee1f51c3, c357ceebcd904ba88f8c1d2bc70100da; no voters:
I20260806 02:12:27.196954 20394 raft_consensus.cc:2781] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:27.197165 20394 raft_consensus.cc:686] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [term 1 LEADER]: Becoming Leader. State: Replica: 4e228b1005184465982a2a11ee1f51c3, State: Running, Role: LEADER
I20260806 02:12:27.197408 20394 consensus_queue.cc:237] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } }
I20260806 02:12:27.199295 20431 tablet_replica.cc:439] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3: TxnStatusTablet state changed. Reason: New leader 4e228b1005184465982a2a11ee1f51c3. Latest consensus state: current_term: 1 leader_uuid: "4e228b1005184465982a2a11ee1f51c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } } }
I20260806 02:12:27.199401 20431 tablet_replica.cc:442] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3: This TxnStatusTablet replica's current role is: LEADER
I20260806 02:12:27.199800 20433 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 02:12:27.200052 20433 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 02:12:27.199987 19864 catalog_manager.cc:5495] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e228b1005184465982a2a11ee1f51c3 (127.18.70.66). New cstate: current_term: 1 leader_uuid: "4e228b1005184465982a2a11ee1f51c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } health_report { overall_health: UNKNOWN } } }
I20260806 02:12:27.219477 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.215804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:33111 (local address 127.0.0.1:50224)
0806 02:12:27.215941 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:27.215958 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:27.216074 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:27.216390 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:27.216393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:27.216401 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 02:12:27.216508 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.216514 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.217235 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.217238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.217842 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.217850 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.218283 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.218286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.218410 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.218502 (+ 92us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 02:12:27.219153 (+ 651us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 02:12:27.219194 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 02:12:27.219279 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 02:12:27.219364 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 02:12:27.219477 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.215921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50224 (local address 127.18.70.66:33111)
0806 02:12:27.216058 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:27.216062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:27.216076 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:12:27.216195 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:27.216197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:27.216250 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:27.216378 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:27.216628 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.217115 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.217963 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.218173 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.218279 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.218660 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:27.219049 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:27.219180 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:27.219293 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 02:12:27.219344 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 02:12:27.222393 20028 raft_consensus.cc:1261] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29 [term 1 FOLLOWER]: Refusing update from remote peer 4e228b1005184465982a2a11ee1f51c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:27.222584 20316 raft_consensus.cc:1261] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da [term 1 FOLLOWER]: Refusing update from remote peer 4e228b1005184465982a2a11ee1f51c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:27.222885 20431 consensus_queue.cc:1035] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:27.223160 20394 consensus_queue.cc:1035] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:27.224650 20395 tablet_replica.cc:439] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29: TxnStatusTablet state changed. Reason: New leader 4e228b1005184465982a2a11ee1f51c3. Latest consensus state: current_term: 1 leader_uuid: "4e228b1005184465982a2a11ee1f51c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } } }
I20260806 02:12:27.224768 20395 tablet_replica.cc:442] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:27.225023 20397 tablet_replica.cc:439] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da: TxnStatusTablet state changed. Reason: New leader 4e228b1005184465982a2a11ee1f51c3. Latest consensus state: current_term: 1 leader_uuid: "4e228b1005184465982a2a11ee1f51c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } } }
I20260806 02:12:27.225142 20397 tablet_replica.cc:442] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:27.227095 20431 tablet_replica.cc:439] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4e228b1005184465982a2a11ee1f51c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } } }
I20260806 02:12:27.227200 20431 tablet_replica.cc:442] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3: This TxnStatusTablet replica's current role is: LEADER
I20260806 02:12:27.227515 20431 tablet_replica.cc:439] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4e228b1005184465982a2a11ee1f51c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } } }
I20260806 02:12:27.227630 20431 tablet_replica.cc:442] T ee47f7c30ef448498ae9f151a52f87ed P 4e228b1005184465982a2a11ee1f51c3: This TxnStatusTablet replica's current role is: LEADER
I20260806 02:12:27.228094 20395 tablet_replica.cc:439] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4e228b1005184465982a2a11ee1f51c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } } }
I20260806 02:12:27.228200 20395 tablet_replica.cc:442] T ee47f7c30ef448498ae9f151a52f87ed P b033bb2eea6244ca942d3e31a6fd2b29: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:27.229679 20397 tablet_replica.cc:439] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4e228b1005184465982a2a11ee1f51c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b033bb2eea6244ca942d3e31a6fd2b29" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39993 } } peers { permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 } } peers { permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 } } }
I20260806 02:12:27.229786 20397 tablet_replica.cc:442] T ee47f7c30ef448498ae9f151a52f87ed P c357ceebcd904ba88f8c1d2bc70100da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:27.237450 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.233909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60106 (local address 127.18.70.65:39993)
0806 02:12:27.234015 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:27.234018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:27.234054 (+ 36us) server_negotiation.cc:407] Connection header received
0806 02:12:27.234228 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:27.234231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:27.234293 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:27.234397 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:27.234645 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.235146 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.236047 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.236281 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.236382 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.236647 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:27.237060 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:27.237178 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:27.237271 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 02:12:27.237323 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 02:12:27.248287 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.244466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:49851 (local address 127.18.70.66:33111)
0806 02:12:27.244591 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:27.244595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:27.244769 (+ 174us) server_negotiation.cc:407] Connection header received
0806 02:12:27.244870 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:27.244872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:27.244909 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:27.244968 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:27.245212 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.245689 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.246947 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.247295 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.247369 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.247802 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:27.247945 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:27.248003 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:27.248142 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 02:12:27.248188 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 02:12:27.248371 20440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.244392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:33111 (local address 127.18.70.65:49851)
0806 02:12:27.244684 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:27.244698 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:27.244795 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:27.245000 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:27.245003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:27.245011 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 02:12:27.245107 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.245112 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.245791 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.245794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.246792 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.246804 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.247480 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.247483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.247599 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.247646 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 02:12:27.248055 (+ 409us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 02:12:27.248062 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 02:12:27.248133 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 02:12:27.248215 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":114,"threads_started":1}
I20260806 02:12:27.255052 20172 raft_consensus.cc:1261] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3 [term 1 FOLLOWER]: Refusing update from remote peer b033bb2eea6244ca942d3e31a6fd2b29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:27.255395 20316 raft_consensus.cc:1261] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da [term 1 FOLLOWER]: Refusing update from remote peer b033bb2eea6244ca942d3e31a6fd2b29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:27.255542 20395 consensus_queue.cc:1035] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e228b1005184465982a2a11ee1f51c3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 33111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:27.255780 20418 consensus_queue.cc:1035] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c357ceebcd904ba88f8c1d2bc70100da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:27.261276 20400 mvcc.cc:205] Tried to move back new op lower bound from 7315383694352318464 to 7315383693552066560. Current Snapshot: MvccSnapshot[applied={T|T < 7315383694352318464}]
I20260806 02:12:27.279666 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.275561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:34751 (local address 127.18.70.65:39993)
0806 02:12:27.275832 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:27.275838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:27.275992 (+ 154us) server_negotiation.cc:407] Connection header received
0806 02:12:27.276109 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:27.276112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:27.276163 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:27.276226 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:27.276480 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.277154 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.278391 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.278726 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.278843 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.279216 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:27.279284 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:27.279341 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:27.279455 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 02:12:27.279494 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260806 02:12:27.279973 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.275487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:39993 (local address 127.18.70.66:34751)
0806 02:12:27.275898 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:27.275913 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:27.276017 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:27.276259 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:27.276263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:27.276270 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 02:12:27.276396 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.276403 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.277257 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.277260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.278258 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:27.278270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.278925 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.278928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:27.279030 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.279084 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 02:12:27.279369 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 02:12:27.279375 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 02:12:27.279447 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 02:12:27.279527 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":137,"threads_started":1}
I20260806 02:12:27.304625 19864 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:27.304803 19864 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:27.306809 19864 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b55f35e6a23f4822a7082cec72302832: Sending DeleteTablet for 3 replicas of tablet 569bd5aaa2b84c319842f89d6b08f751
I20260806 02:12:27.307303 20006 tablet_service.cc:1514] Processing DeleteTablet for tablet 569bd5aaa2b84c319842f89d6b08f751 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:27 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:60096
I20260806 02:12:27.307475 20411 tablet_replica.cc:330] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29: stopping tablet replica
I20260806 02:12:27.307627 20411 raft_consensus.cc:2229] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:12:27.307721 20296 tablet_service.cc:1514] Processing DeleteTablet for tablet 569bd5aaa2b84c319842f89d6b08f751 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:27 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:44302
I20260806 02:12:27.307891 20412 tablet_replica.cc:330] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da: stopping tablet replica
I20260806 02:12:27.307928 20411 raft_consensus.cc:2258] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:27.308001 20412 raft_consensus.cc:2229] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:27.308123 20412 raft_consensus.cc:2258] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:27.308615 20152 tablet_service.cc:1514] Processing DeleteTablet for tablet 569bd5aaa2b84c319842f89d6b08f751 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:27 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:50222
I20260806 02:12:27.308794 20410 tablet_replica.cc:330] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3: stopping tablet replica
I20260806 02:12:27.308890 20410 raft_consensus.cc:2229] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:27.308992 20410 raft_consensus.cc:2258] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:27.309033 20412 ts_tablet_manager.cc:1905] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:27.309831 20410 ts_tablet_manager.cc:1905] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:27.312100 20412 ts_tablet_manager.cc:1918] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:12:27.312179 20412 log.cc:1196] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/569bd5aaa2b84c319842f89d6b08f751
I20260806 02:12:27.312314 20411 ts_tablet_manager.cc:1905] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:27.312459 20412 ts_tablet_manager.cc:1939] T 569bd5aaa2b84c319842f89d6b08f751 P c357ceebcd904ba88f8c1d2bc70100da: Deleting consensus metadata
I20260806 02:12:27.313210 19853 catalog_manager.cc:4841] TS c357ceebcd904ba88f8c1d2bc70100da (127.18.70.67:42595): tablet 569bd5aaa2b84c319842f89d6b08f751 (table test-table [id=6b3a954a79094aba965edb0a2cdfe570]) successfully deleted
I20260806 02:12:27.314682 20411 ts_tablet_manager.cc:1918] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:12:27.314734 20411 log.cc:1196] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-0/wal/wals/569bd5aaa2b84c319842f89d6b08f751
I20260806 02:12:27.314916 20411 ts_tablet_manager.cc:1939] T 569bd5aaa2b84c319842f89d6b08f751 P b033bb2eea6244ca942d3e31a6fd2b29: Deleting consensus metadata
I20260806 02:12:27.316663 20410 ts_tablet_manager.cc:1918] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:12:27.316764 20410 log.cc:1196] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/569bd5aaa2b84c319842f89d6b08f751
I20260806 02:12:27.317063 20410 ts_tablet_manager.cc:1939] T 569bd5aaa2b84c319842f89d6b08f751 P 4e228b1005184465982a2a11ee1f51c3: Deleting consensus metadata
I20260806 02:12:27.317261 19850 catalog_manager.cc:4841] TS b033bb2eea6244ca942d3e31a6fd2b29 (127.18.70.65:39993): tablet 569bd5aaa2b84c319842f89d6b08f751 (table test-table [id=6b3a954a79094aba965edb0a2cdfe570]) successfully deleted
I20260806 02:12:27.317811 19851 catalog_manager.cc:4841] TS 4e228b1005184465982a2a11ee1f51c3 (127.18.70.66:33111): tablet 569bd5aaa2b84c319842f89d6b08f751 (table test-table [id=6b3a954a79094aba965edb0a2cdfe570]) successfully deleted
Aug 06 02:12:27 dist-test-slave-6k22 krb5kdc[19819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982346, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.70.65@KRBTEST.COM
I20260806 02:12:27.328845 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.317886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60112 (local address 127.18.70.65:39993)
0806 02:12:27.318064 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:27.318068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:27.318566 (+ 498us) server_negotiation.cc:407] Connection header received
0806 02:12:27.318727 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:27.318730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:27.318786 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:27.318880 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:27.319643 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.320263 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.321200 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.321421 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.321537 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.325101 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:27.325114 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:27.325129 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:27.325154 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:27.326962 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:27.327428 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:27.327435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:27.327444 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:27.327527 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:27.327889 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:27.327893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:27.327895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:27.328146 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:27.328403 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:27.328549 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 02:12:27.328693 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260806 02:12:27.329528 20070 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60112
I20260806 02:12:27.338737 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:27.331108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35842 (local address 127.18.70.126:39215)
0806 02:12:27.331244 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:27.331250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:27.331697 (+ 447us) server_negotiation.cc:407] Connection header received
0806 02:12:27.331805 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:27.331808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:27.331855 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:27.331951 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:27.332566 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.333084 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.333866 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:27.334102 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:27.334222 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:27.335504 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:27.335508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:27.335510 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:27.335535 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:27.337037 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:27.337483 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:27.337487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:27.337489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:27.337577 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:27.337861 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:27.337865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:27.337867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:27.338043 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:27.338186 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:27.338454 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 02:12:27.338573 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 02:12:27.339296 19864 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35842
I20260806 02:12:27.340689 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 19940
I20260806 02:12:27.352591 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 20086
I20260806 02:12:27.361418 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 20230
I20260806 02:12:27.369237 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 19833
2026-08-06T02:12:27Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2838 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:12:27 dist-test-slave-6k22 krb5kdc[20455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:12:27 dist-test-slave-6k22 krb5kdc[20455](info): set up 2 sockets
Aug 06 02:12:27 dist-test-slave-6k22 krb5kdc[20455](info): commencing operation
krb5kdc: starting...
W20260806 02:12:28.453666 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:12:28 dist-test-slave-6k22 krb5kdc[20455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:12:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:12:28.578658 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:42113
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:42113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:28.690173 20468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:28.690457 20468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:28.690538 20468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:28.694563 20468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:28.694645 20468 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 02:12:28.694679 20468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:28.694718 20468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:28.694756 20468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:28.694792 20468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:28.699944 20468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:42113
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:42113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:28.700978 20468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:28.701979 20468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:28.710522 20477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:28.710530 20473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:28.710529 20474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:28.710785 20468 server_base.cc:1034] running on GCE node
I20260806 02:12:28.711334 20468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:28.712400 20468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:28.713624 20468 hybrid_clock.cc:648] HybridClock initialized: now 1785982348713605 us; error 36 us; skew 500 ppm
Aug 06 02:12:28 dist-test-slave-6k22 krb5kdc[20455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982348, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:28.716465 20468 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:12:28.717283 20468 webserver.cc:460] Webserver started at http://127.18.70.126:44621/ using document root <none> and password file <none>
I20260806 02:12:28.717782 20468 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:28.717837 20468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:28.717995 20468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:28.720089 20468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "e27fa7ffa8a8472b9819c1dee9e1723d"
format_stamp: "Formatted at 2026-08-06 02:12:28 on dist-test-slave-6k22"
server_key: "0770691fa776a9bfd74fae651ad333ce"
server_key_iv: "ffaf56ef63b8f2ae85bd3a0d337f7015"
server_key_version: "encryptionkey@0"
I20260806 02:12:28.720620 20468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "e27fa7ffa8a8472b9819c1dee9e1723d"
format_stamp: "Formatted at 2026-08-06 02:12:28 on dist-test-slave-6k22"
server_key: "0770691fa776a9bfd74fae651ad333ce"
server_key_iv: "ffaf56ef63b8f2ae85bd3a0d337f7015"
server_key_version: "encryptionkey@0"
I20260806 02:12:28.724411 20468 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 02:12:28.726873 20484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:28.728154 20468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:28.728262 20468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "e27fa7ffa8a8472b9819c1dee9e1723d"
format_stamp: "Formatted at 2026-08-06 02:12:28 on dist-test-slave-6k22"
server_key: "0770691fa776a9bfd74fae651ad333ce"
server_key_iv: "ffaf56ef63b8f2ae85bd3a0d337f7015"
server_key_version: "encryptionkey@0"
I20260806 02:12:28.728346 20468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:28.741441 20468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:28.744071 20468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:28.744225 20468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:28.751946 20468 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:42113
I20260806 02:12:28.751973 20549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:42113 every 8 connection(s)
I20260806 02:12:28.752820 20468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:28.754832 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 20468
I20260806 02:12:28.754922 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:28.755204 18713 external_mini_cluster.cc:1428] Setting key 2d5a43358d5c8395fd65844f30f919e4
I20260806 02:12:28.755664 20550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:28.761623 20550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d: Bootstrap starting.
Aug 06 02:12:28 dist-test-slave-6k22 krb5kdc[20455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:28.764343 20550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:28.765216 20550 log.cc:824] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:28.767419 20550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d: No bootstrap required, opened a new log
I20260806 02:12:28.769949 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:28.756626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39828 (local address 127.18.70.126:42113)
0806 02:12:28.757057 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:28.757072 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:28.757106 (+ 34us) server_negotiation.cc:407] Connection header received
0806 02:12:28.757773 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:28.757788 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:28.758089 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:28.758437 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:28.759254 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:28.760292 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:28.761134 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:28.761594 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:28.761762 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:28.764557 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:28.764562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:28.764575 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:28.764602 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:28.767081 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:28.767629 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:28.767635 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:28.767641 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:28.767785 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:28.768088 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:28.768091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:28.768092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:28.768473 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:28.768619 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:28.768923 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 02:12:28.769222 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":142,"threads_started":1}
I20260806 02:12:28.771039 20550 raft_consensus.cc:348] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27fa7ffa8a8472b9819c1dee9e1723d" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42113 } }
I20260806 02:12:28.771288 20550 raft_consensus.cc:374] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:28.771365 20550 raft_consensus.cc:729] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e27fa7ffa8a8472b9819c1dee9e1723d, State: Initialized, Role: FOLLOWER
I20260806 02:12:28.771883 20550 consensus_queue.cc:260] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27fa7ffa8a8472b9819c1dee9e1723d" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42113 } }
I20260806 02:12:28.772037 20550 raft_consensus.cc:388] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:28.772105 20550 raft_consensus.cc:482] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:28.772218 20550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:28.773214 20550 raft_consensus.cc:504] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27fa7ffa8a8472b9819c1dee9e1723d" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42113 } }
I20260806 02:12:28.773512 20550 leader_election.cc:302] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e27fa7ffa8a8472b9819c1dee9e1723d; no voters:
I20260806 02:12:28.773757 20550 leader_election.cc:288] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:28.773891 20555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:28.774125 20555 raft_consensus.cc:686] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [term 1 LEADER]: Becoming Leader. State: Replica: e27fa7ffa8a8472b9819c1dee9e1723d, State: Running, Role: LEADER
I20260806 02:12:28.774441 20555 consensus_queue.cc:237] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27fa7ffa8a8472b9819c1dee9e1723d" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42113 } }
I20260806 02:12:28.774971 20550 sys_catalog.cc:564] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:28.776269 20556 sys_catalog.cc:455] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e27fa7ffa8a8472b9819c1dee9e1723d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27fa7ffa8a8472b9819c1dee9e1723d" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42113 } } }
I20260806 02:12:28.776249 20557 sys_catalog.cc:455] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e27fa7ffa8a8472b9819c1dee9e1723d. Latest consensus state: current_term: 1 leader_uuid: "e27fa7ffa8a8472b9819c1dee9e1723d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e27fa7ffa8a8472b9819c1dee9e1723d" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42113 } } }
I20260806 02:12:28.776407 20557 sys_catalog.cc:458] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:28.776407 20556 sys_catalog.cc:458] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:28.776804 20564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:28.779706 20564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:28.785181 20564 catalog_manager.cc:1347] Generated new cluster ID: 0bd26439fbc84885abf4050365baa787
I20260806 02:12:28.785259 20564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:28.797328 20564 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:28.798607 20564 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:28.805665 20564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d: Generated new TSK 0
I20260806 02:12:28.806425 20564 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:12:28.842935 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42113
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:28.957276 20577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:28.957508 20577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:28.957578 20577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:28.961606 20577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:28.961671 20577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:28.961753 20577 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 02:12:28.961791 20577 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 02:12:28.961846 20577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:12:28.966902 20577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.70.126:42113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:28.967962 20577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:28.969017 20577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:28.978431 20583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:28.978431 20582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:28.978509 20585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:28.978638 20577 server_base.cc:1034] running on GCE node
I20260806 02:12:28.979185 20577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:28.979936 20577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:28.981127 20577 hybrid_clock.cc:648] HybridClock initialized: now 1785982348981088 us; error 51 us; skew 500 ppm
Aug 06 02:12:28 dist-test-slave-6k22 krb5kdc[20455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982348, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:28.983976 20577 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:12:28.984833 20577 webserver.cc:460] Webserver started at http://127.18.70.65:42967/ using document root <none> and password file <none>
I20260806 02:12:28.985421 20577 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:28.985525 20577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:28.985757 20577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:28.987839 20577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "45bf89be760448f39b7368267db96f98"
format_stamp: "Formatted at 2026-08-06 02:12:28 on dist-test-slave-6k22"
server_key: "e4f0df3e29a42976a5e34b7d8cd14840"
server_key_iv: "8a2edf290dcffc26961bbf323a426292"
server_key_version: "encryptionkey@0"
I20260806 02:12:28.988417 20577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "45bf89be760448f39b7368267db96f98"
format_stamp: "Formatted at 2026-08-06 02:12:28 on dist-test-slave-6k22"
server_key: "e4f0df3e29a42976a5e34b7d8cd14840"
server_key_iv: "8a2edf290dcffc26961bbf323a426292"
server_key_version: "encryptionkey@0"
I20260806 02:12:28.992228 20577 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 02:12:28.994771 20592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:28.996191 20577 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:28.996341 20577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "45bf89be760448f39b7368267db96f98"
format_stamp: "Formatted at 2026-08-06 02:12:28 on dist-test-slave-6k22"
server_key: "e4f0df3e29a42976a5e34b7d8cd14840"
server_key_iv: "8a2edf290dcffc26961bbf323a426292"
server_key_version: "encryptionkey@0"
I20260806 02:12:28.996455 20577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:29.019834 20577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:29.023154 20577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:29.023341 20577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:29.025017 20577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:29.025066 20577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:29.025106 20577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:29.025153 20577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 02:12:29 dist-test-slave-6k22 krb5kdc[20455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982348, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:29.035735 20577 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:34163
I20260806 02:12:29.036084 20716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:34163 every 8 connection(s)
I20260806 02:12:29.036659 20577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:12:29.039062 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.025766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:54189 (local address 127.18.70.126:42113)
0806 02:12:29.025911 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.025916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.026789 (+ 873us) server_negotiation.cc:407] Connection header received
0806 02:12:29.027852 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.027856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.027924 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.027991 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:29.029386 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.030028 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.031824 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.032147 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.032259 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.035296 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:29.035300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:29.035303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:29.035337 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:29.037191 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.037736 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.037738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.037739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.037834 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.038130 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.038133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.038133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.038305 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:29.038396 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.038834 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.038941 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 02:12:29.039835 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 20577
I20260806 02:12:29.039949 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:12:29.039947 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.026192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42113 (local address 127.18.70.65:54189)
0806 02:12:29.026621 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.026671 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.027601 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.028363 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.028369 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.028777 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:29.029239 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.029252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.030165 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.030175 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.031708 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.031721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.032390 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.032395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.032554 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.033644 (+ 1090us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:29.033661 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:29.035120 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:29.037334 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.037340 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.037352 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.037627 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.037927 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.037929 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.037930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.038050 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.038421 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:29.038425 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:29.038627 (+ 202us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.038778 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.038998 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":117,"threads_started":1}
I20260806 02:12:29.040258 18713 external_mini_cluster.cc:1428] Setting key cedaf514038e035c8fc96157a6fb626a
I20260806 02:12:29.041448 20717 heartbeater.cc:347] Connected to a master server at 127.18.70.126:42113
I20260806 02:12:29.041601 20717 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:29.042160 20717 heartbeater.cc:511] Master 127.18.70.126:42113 requested a full tablet report, sending...
I20260806 02:12:29.044150 20501 ts_manager.cc:194] Registered new tserver with Master: 45bf89be760448f39b7368267db96f98 (127.18.70.65:34163)
I20260806 02:12:29.045696 20501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:54189
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
I20260806 02:12:29.052372 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.044289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:33543 (local address 127.18.70.126:42113)
0806 02:12:29.044445 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.044451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.044695 (+ 244us) server_negotiation.cc:407] Connection header received
0806 02:12:29.044773 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.044777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.044833 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.044939 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:29.045646 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.046366 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.047370 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.047715 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.047840 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.049071 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:29.049075 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:29.049077 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:29.049101 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:29.050589 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.051105 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.051109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.051111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.051196 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.051481 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.051486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.051488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.051727 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:29.051854 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.052108 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.052220 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 02:12:29.052392 20724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.043919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42113 (local address 127.18.70.65:33543)
0806 02:12:29.044564 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.044579 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.044688 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.045085 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.045091 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.045336 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:29.045497 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.045509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.046508 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.046510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.047222 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.047232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.047844 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.047848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.047969 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.048549 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:29.048579 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:29.048917 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:29.050772 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.050776 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.050778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.050979 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.051292 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.051295 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.051297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.051387 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.051846 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:29.051849 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:29.051947 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.052055 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.052208 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":2,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:12:29.071113 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42113
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:29.183276 20726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:29.183499 20726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:29.183543 20726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:29.187579 20726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:29.187630 20726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:29.187667 20726 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 02:12:29.187681 20726 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 02:12:29.187741 20726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:12:29.192589 20726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.70.126:42113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:29.193660 20726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:29.194665 20726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:29.203872 20734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:29.203905 20731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:29.203899 20732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:29.204114 20726 server_base.cc:1034] running on GCE node
I20260806 02:12:29.204523 20726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:29.205147 20726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:29.206328 20726 hybrid_clock.cc:648] HybridClock initialized: now 1785982349206296 us; error 46 us; skew 500 ppm
Aug 06 02:12:29 dist-test-slave-6k22 krb5kdc[20455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982349, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:29.208994 20726 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:12:29.209874 20726 webserver.cc:460] Webserver started at http://127.18.70.66:46673/ using document root <none> and password file <none>
I20260806 02:12:29.210408 20726 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:29.210496 20726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:29.210724 20726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:29.212698 20726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "9145b72ee2554e6eaf55fec53d59a7fd"
format_stamp: "Formatted at 2026-08-06 02:12:29 on dist-test-slave-6k22"
server_key: "077bab072861edd94f4a1ca2ad9357bc"
server_key_iv: "65618b44336b96a2696e1a339c92a0fa"
server_key_version: "encryptionkey@0"
I20260806 02:12:29.213263 20726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "9145b72ee2554e6eaf55fec53d59a7fd"
format_stamp: "Formatted at 2026-08-06 02:12:29 on dist-test-slave-6k22"
server_key: "077bab072861edd94f4a1ca2ad9357bc"
server_key_iv: "65618b44336b96a2696e1a339c92a0fa"
server_key_version: "encryptionkey@0"
I20260806 02:12:29.216758 20726 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:12:29.219034 20742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:29.220394 20726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:29.220534 20726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "9145b72ee2554e6eaf55fec53d59a7fd"
format_stamp: "Formatted at 2026-08-06 02:12:29 on dist-test-slave-6k22"
server_key: "077bab072861edd94f4a1ca2ad9357bc"
server_key_iv: "65618b44336b96a2696e1a339c92a0fa"
server_key_version: "encryptionkey@0"
I20260806 02:12:29.220643 20726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:29.242954 20726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:29.245783 20726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:29.245975 20726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:29.247473 20726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:29.247546 20726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:29.247648 20726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:29.247715 20726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 02:12:29 dist-test-slave-6k22 krb5kdc[20455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982349, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:29.258848 20726 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:41165
I20260806 02:12:29.258944 20863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:41165 every 8 connection(s)
I20260806 02:12:29.259802 20726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:12:29.261955 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.248399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:40763 (local address 127.18.70.126:42113)
0806 02:12:29.248576 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.248583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.249382 (+ 799us) server_negotiation.cc:407] Connection header received
0806 02:12:29.250448 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.250452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.250512 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.250596 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:29.252278 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.252994 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.254787 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.255111 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.255260 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.258265 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:29.258269 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:29.258272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:29.258298 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:29.260031 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.260641 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.260645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.260647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.260746 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.261065 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.261067 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.261068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.261240 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:29.261315 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.261742 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.261838 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 02:12:29.262712 20751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.248764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42113 (local address 127.18.70.66:40763)
0806 02:12:29.249208 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.249258 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.250215 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.250817 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.250831 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.251358 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:29.252081 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.252105 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.253104 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.253111 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.254655 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.254668 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.255299 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.255305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.255541 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.256132 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:29.256163 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:29.258084 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:29.260162 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.260168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.260181 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.260545 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.260841 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.260843 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.260844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.260976 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.261329 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:29.261333 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:29.261536 (+ 203us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.261687 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.261907 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":126,"threads_started":1}
I20260806 02:12:29.263913 20864 heartbeater.cc:347] Connected to a master server at 127.18.70.126:42113
I20260806 02:12:29.264072 20864 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:29.264557 20864 heartbeater.cc:511] Master 127.18.70.126:42113 requested a full tablet report, sending...
I20260806 02:12:29.265552 20502 ts_manager.cc:194] Registered new tserver with Master: 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66:41165)
I20260806 02:12:29.266217 20502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:40763
I20260806 02:12:29.267632 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 20726
I20260806 02:12:29.267748 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:12:29.268054 18713 external_mini_cluster.cc:1428] Setting key 2d51812d024bc7f36560368887b97d96
I20260806 02:12:29.273525 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.266121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:58993 (local address 127.18.70.126:42113)
0806 02:12:29.266245 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.266262 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.266575 (+ 313us) server_negotiation.cc:407] Connection header received
0806 02:12:29.266684 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.266687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.266732 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.266819 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:29.267253 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.267909 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.268870 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.269091 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.269193 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.270392 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:29.270395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:29.270397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:29.270417 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:29.271814 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.272318 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.272321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.272322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.272397 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.272720 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.272722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.272723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.272931 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:29.273008 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.273324 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.273413 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 02:12:29.273749 20871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.266056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42113 (local address 127.18.70.66:58993)
0806 02:12:29.266496 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.266507 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.266610 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.266835 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.266838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.267019 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:29.267120 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.267125 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.268032 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.268035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.268758 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.268768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.269206 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.269210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.269331 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.269796 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:29.269811 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:29.270251 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:29.271922 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.271927 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.271930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.272188 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.272516 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.272521 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.272524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.272628 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.273028 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:29.273034 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:29.273155 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.273275 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.273442 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:12:29.298070 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42113
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:29.411801 20874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:29.412094 20874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:29.412168 20874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:29.416162 20874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:29.416241 20874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:29.416316 20874 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 02:12:29.416404 20874 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 02:12:29.416464 20874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:12:29.421649 20874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.70.126:42113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:29.422720 20874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:29.424156 20874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:29.433879 20880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:29.433878 20882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:29.433902 20879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:29.434172 20874 server_base.cc:1034] running on GCE node
I20260806 02:12:29.434640 20874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:29.435282 20874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:29.436462 20874 hybrid_clock.cc:648] HybridClock initialized: now 1785982349436423 us; error 50 us; skew 500 ppm
Aug 06 02:12:29 dist-test-slave-6k22 krb5kdc[20455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982349, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:29.439043 20874 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:12:29.439961 20874 webserver.cc:460] Webserver started at http://127.18.70.67:45731/ using document root <none> and password file <none>
I20260806 02:12:29.440527 20874 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:29.440614 20874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:29.440838 20874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:29.442852 20874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "f42bbabc6cd44ff5b16e46849184a3e4"
format_stamp: "Formatted at 2026-08-06 02:12:29 on dist-test-slave-6k22"
server_key: "8de5dec00d82f336c64d88c9917e80c1"
server_key_iv: "6420afb47b28cdcf2a2ff4aedc8fe7ec"
server_key_version: "encryptionkey@0"
I20260806 02:12:29.443437 20874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "f42bbabc6cd44ff5b16e46849184a3e4"
format_stamp: "Formatted at 2026-08-06 02:12:29 on dist-test-slave-6k22"
server_key: "8de5dec00d82f336c64d88c9917e80c1"
server_key_iv: "6420afb47b28cdcf2a2ff4aedc8fe7ec"
server_key_version: "encryptionkey@0"
I20260806 02:12:29.447304 20874 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 02:12:29.449612 20889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:29.450766 20874 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 02:12:29.450901 20874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "f42bbabc6cd44ff5b16e46849184a3e4"
format_stamp: "Formatted at 2026-08-06 02:12:29 on dist-test-slave-6k22"
server_key: "8de5dec00d82f336c64d88c9917e80c1"
server_key_iv: "6420afb47b28cdcf2a2ff4aedc8fe7ec"
server_key_version: "encryptionkey@0"
I20260806 02:12:29.451009 20874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:29.481034 20874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:29.484071 20874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:29.484279 20874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:29.485888 20874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:29.485955 20874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:29.486027 20874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:29.486066 20874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 02:12:29 dist-test-slave-6k22 krb5kdc[20455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982349, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:29.497213 20874 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:36567
I20260806 02:12:29.497538 21006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:36567 every 8 connection(s)
I20260806 02:12:29.498246 20874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:12:29.500485 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.486673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:45129 (local address 127.18.70.126:42113)
0806 02:12:29.486846 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.486852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.488210 (+ 1358us) server_negotiation.cc:407] Connection header received
0806 02:12:29.489703 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.489706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.489762 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.489853 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:29.491608 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.492259 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.493567 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.493931 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.494054 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.496638 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:29.496641 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:29.496645 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:29.496672 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:29.498218 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.498727 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.498732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.498734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.498834 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.499262 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.499268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.499271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.499509 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:29.499692 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.500257 (+ 565us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.500355 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 02:12:29.501322 20898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.487513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42113 (local address 127.18.70.67:45129)
0806 02:12:29.488026 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.488088 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.489411 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.490076 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.490087 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.490625 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:29.491409 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.491433 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.492407 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.492411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.493423 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.493433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.494096 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.494101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.494304 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.494910 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:29.494933 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:29.496467 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:29.498340 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.498346 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.498374 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.498617 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.498948 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.498952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.498954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.499130 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.499691 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:29.499696 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:29.500028 (+ 332us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.500188 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.500430 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":95,"threads_started":1}
I20260806 02:12:29.502540 21007 heartbeater.cc:347] Connected to a master server at 127.18.70.126:42113
I20260806 02:12:29.502712 21007 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:29.503192 21007 heartbeater.cc:511] Master 127.18.70.126:42113 requested a full tablet report, sending...
I20260806 02:12:29.504406 20502 ts_manager.cc:194] Registered new tserver with Master: f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567)
I20260806 02:12:29.505180 20502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:45129
I20260806 02:12:29.505191 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 20874
I20260806 02:12:29.505335 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:12:29.505674 18713 external_mini_cluster.cc:1428] Setting key a7cff4ea27a8d91cec67a2e3bb54aaeb
I20260806 02:12:29.507807 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:12:29.512876 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.505511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:37261 (local address 127.18.70.126:42113)
0806 02:12:29.505615 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.505619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.505631 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:12:29.505665 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.505668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.505706 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.505785 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:29.506301 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.507045 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.508163 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.508507 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.508630 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.509602 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:29.509608 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:29.509614 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:29.509647 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:29.511178 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.511697 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.511701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.511702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.511791 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.512068 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.512073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.512075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.512240 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:29.512356 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.512616 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.512723 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 02:12:29.512894 21013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.504696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42113 (local address 127.18.70.67:37261)
0806 02:12:29.504974 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.504989 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.505068 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.505890 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.505893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.506047 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:29.506174 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.506182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.507198 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.507202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.508050 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.508061 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.508619 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.508622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.508691 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.509159 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:29.509170 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:29.509465 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:29.511294 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.511300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.511303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.511600 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.511876 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.511881 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.511883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.511972 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.512348 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:29.512351 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:29.512449 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.512558 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.512698 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":73,"threads_started":1}
Aug 06 02:12:29 dist-test-slave-6k22 krb5kdc[20455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 02:12:29 dist-test-slave-6k22 krb5kdc[20455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:29.534993 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.527175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39842 (local address 127.18.70.126:42113)
0806 02:12:29.527301 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.527304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.527316 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:12:29.527351 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.527353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.527388 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.527458 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:29.528174 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.528654 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.529419 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.529713 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.529792 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.532089 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:29.532092 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:29.532094 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:29.532115 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:29.533584 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.533942 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.533944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.533945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.534015 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.534238 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.534240 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.534249 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.534401 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:29.534498 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.534686 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.534773 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 02:12:29.537703 20502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:12:29.540684 20502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:29.555121 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.548879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:36567 (local address 127.0.0.1:42598)
0806 02:12:29.549754 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.549820 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.549952 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.550762 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.550769 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.550790 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:29.550879 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.550885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.552275 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.552278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.553462 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.553472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.554541 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.554545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.554667 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.554710 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.554820 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.554961 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":35,"spinlock_wait_cycles":5248,"thread_start_us":96,"threads_started":1}
I20260806 02:12:29.555447 21025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.548781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:34163 (local address 127.0.0.1:51296)
0806 02:12:29.549754 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.549820 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.549952 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.550559 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.550565 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.550587 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:29.550761 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.550769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.552404 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.552406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.553681 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.553693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.554681 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.554684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.554782 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.554817 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.555229 (+ 412us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.555305 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"mutex_wait_us":63,"spinlock_wait_cycles":6400,"thread_start_us":68,"threads_started":1}
I20260806 02:12:29.555763 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.549520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42598 (local address 127.18.70.67:36567)
0806 02:12:29.550174 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.550186 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.550213 (+ 27us) server_negotiation.cc:407] Connection header received
0806 02:12:29.550309 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.550313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.550541 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.550757 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:29.550997 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.552159 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.553589 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.554381 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.554516 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.554706 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.555486 (+ 780us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.555623 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":112,"threads_started":1}
I20260806 02:12:29.556564 21028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.549329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:41165 (local address 127.0.0.1:43976)
0806 02:12:29.550177 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.550188 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.550278 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.550963 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.550966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.550986 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:29.551069 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.551073 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.552133 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.552136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.553681 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.553693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.556103 (+ 2410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.556106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.556203 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.556246 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.556325 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.556428 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":68,"thread_start_us":96,"threads_started":1}
I20260806 02:12:29.556635 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.549512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43976 (local address 127.18.70.66:41165)
0806 02:12:29.549969 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.549983 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.550366 (+ 383us) server_negotiation.cc:407] Connection header received
0806 02:12:29.550446 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.550454 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.550645 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.550822 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:29.551183 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.552003 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.554786 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.555733 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.555848 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.556030 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.556398 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.556501 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":114,"threads_started":1}
I20260806 02:12:29.557034 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.548912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51296 (local address 127.18.70.65:34163)
0806 02:12:29.549606 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.549614 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.550089 (+ 475us) server_negotiation.cc:407] Connection header received
0806 02:12:29.550173 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.550179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.550397 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.550546 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:29.550897 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.552159 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.553817 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.554574 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.554673 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.554821 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.555707 (+ 886us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.555827 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":77,"threads_started":1}
I20260806 02:12:29.558470 20940 tablet_service.cc:1467] Processing CreateTablet for tablet 7c2ede1858414906816c7a3ba3b1e887 (DEFAULT_TABLE table=test-table [id=e4c09fc559a046fdb13aa674472e09d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:29.559373 20940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c2ede1858414906816c7a3ba3b1e887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:29.559394 20796 tablet_service.cc:1467] Processing CreateTablet for tablet 7c2ede1858414906816c7a3ba3b1e887 (DEFAULT_TABLE table=test-table [id=e4c09fc559a046fdb13aa674472e09d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:29.559381 20644 tablet_service.cc:1467] Processing CreateTablet for tablet 7c2ede1858414906816c7a3ba3b1e887 (DEFAULT_TABLE table=test-table [id=e4c09fc559a046fdb13aa674472e09d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:29.560431 20796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c2ede1858414906816c7a3ba3b1e887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:29.560431 20644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c2ede1858414906816c7a3ba3b1e887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:29.566530 21031 tablet_bootstrap.cc:492] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4: Bootstrap starting.
I20260806 02:12:29.568059 21033 tablet_bootstrap.cc:492] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd: Bootstrap starting.
I20260806 02:12:29.568374 21032 tablet_bootstrap.cc:492] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98: Bootstrap starting.
I20260806 02:12:29.569140 21031 tablet_bootstrap.cc:654] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:29.570055 21031 log.cc:824] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:29.570116 21032 tablet_bootstrap.cc:654] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:29.570169 21033 tablet_bootstrap.cc:654] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:29.570950 21032 log.cc:824] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:29.571009 21033 log.cc:824] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:29.572386 21031 tablet_bootstrap.cc:492] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4: No bootstrap required, opened a new log
I20260806 02:12:29.572592 21031 ts_tablet_manager.cc:1397] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260806 02:12:29.572791 21032 tablet_bootstrap.cc:492] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98: No bootstrap required, opened a new log
I20260806 02:12:29.572983 21032 ts_tablet_manager.cc:1397] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 02:12:29.573066 21033 tablet_bootstrap.cc:492] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd: No bootstrap required, opened a new log
I20260806 02:12:29.573246 21033 ts_tablet_manager.cc:1397] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 02:12:29.576550 21032 raft_consensus.cc:348] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } }
I20260806 02:12:29.576614 21031 raft_consensus.cc:348] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } }
I20260806 02:12:29.576906 21031 raft_consensus.cc:374] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:29.576910 21032 raft_consensus.cc:374] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:29.576967 21031 raft_consensus.cc:729] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42bbabc6cd44ff5b16e46849184a3e4, State: Initialized, Role: FOLLOWER
I20260806 02:12:29.576974 21032 raft_consensus.cc:729] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45bf89be760448f39b7368267db96f98, State: Initialized, Role: FOLLOWER
I20260806 02:12:29.577005 21033 raft_consensus.cc:348] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } }
I20260806 02:12:29.577221 21033 raft_consensus.cc:374] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:29.577268 21033 raft_consensus.cc:729] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9145b72ee2554e6eaf55fec53d59a7fd, State: Initialized, Role: FOLLOWER
I20260806 02:12:29.577425 21031 consensus_queue.cc:260] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } }
I20260806 02:12:29.577425 21032 consensus_queue.cc:260] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } }
I20260806 02:12:29.577653 21033 consensus_queue.cc:260] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } }
I20260806 02:12:29.578467 21032 ts_tablet_manager.cc:1428] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 02:12:29.578486 21031 ts_tablet_manager.cc:1428] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 02:12:29.578486 21033 ts_tablet_manager.cc:1428] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 02:12:29.578855 20717 heartbeater.cc:503] Master 127.18.70.126:42113 was elected leader, sending a full tablet report...
I20260806 02:12:29.578860 21007 heartbeater.cc:503] Master 127.18.70.126:42113 was elected leader, sending a full tablet report...
I20260806 02:12:29.579435 20864 heartbeater.cc:503] Master 127.18.70.126:42113 was elected leader, sending a full tablet report...
I20260806 02:12:29.615897 21039 raft_consensus.cc:482] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:29.615885 21037 raft_consensus.cc:482] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:29.616108 21039 raft_consensus.cc:504] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } }
I20260806 02:12:29.616106 21037 raft_consensus.cc:504] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } }
I20260806 02:12:29.617204 21039 leader_election.cc:288] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45bf89be760448f39b7368267db96f98 (127.18.70.65:34163), f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567)
I20260806 02:12:29.617712 21037 leader_election.cc:288] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567), 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66:41165)
I20260806 02:12:29.622148 20751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.617450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:34163 (local address 127.18.70.66:37693)
0806 02:12:29.617741 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.617756 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.617847 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.618333 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.618339 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.618399 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:29.618524 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.618531 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.619528 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.619531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.620766 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.620777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.621648 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.621652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.621805 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.621835 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.621905 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.621979 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":28}
I20260806 02:12:29.622664 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.617584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:53577 (local address 127.18.70.66:41165)
0806 02:12:29.617700 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.617704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.618196 (+ 492us) server_negotiation.cc:407] Connection header received
0806 02:12:29.618442 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.618445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.618496 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.618545 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:29.618966 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.619770 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.621229 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.621936 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.622071 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.622146 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.622471 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.622538 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":37}
I20260806 02:12:29.622818 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.617811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:37693 (local address 127.18.70.65:34163)
0806 02:12:29.618020 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.618025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.618037 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:12:29.618083 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.618086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.618135 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.618207 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:29.618661 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.619406 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.620911 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.621455 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.621558 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.621606 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.622618 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.622692 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260806 02:12:29.622982 21043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.618211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:38293 (local address 127.18.70.67:36567)
0806 02:12:29.618609 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.618618 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.618634 (+ 16us) server_negotiation.cc:407] Connection header received
0806 02:12:29.618684 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.618687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.618731 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.618811 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:29.619156 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.619915 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.621298 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.621921 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.622318 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.622388 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.622757 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.622835 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":300,"thread_start_us":162,"threads_started":1}
I20260806 02:12:29.623003 21040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.617450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:41165 (local address 127.18.70.65:53577)
0806 02:12:29.618109 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.618123 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.618279 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.618680 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.618683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.618705 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:29.618843 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.618852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.619901 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.619903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.621078 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.621089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.622194 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.622197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.622320 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.622345 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.622801 (+ 456us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.622862 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":8,"thread_start_us":144,"threads_started":1}
I20260806 02:12:29.623483 20665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c2ede1858414906816c7a3ba3b1e887" candidate_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45bf89be760448f39b7368267db96f98" is_pre_election: true
I20260806 02:12:29.623550 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.617613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:38305 (local address 127.18.70.67:36567)
0806 02:12:29.617736 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.617754 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.618761 (+ 1007us) server_negotiation.cc:407] Connection header received
0806 02:12:29.618950 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.618954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.619002 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.619052 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:29.619895 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.620599 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.622209 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.622800 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.622897 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.622947 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.623369 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.623444 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:12:29.623517 20818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c2ede1858414906816c7a3ba3b1e887" candidate_uuid: "45bf89be760448f39b7368267db96f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" is_pre_election: true
I20260806 02:12:29.623731 20818 raft_consensus.cc:2454] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45bf89be760448f39b7368267db96f98 in term 0.
I20260806 02:12:29.623759 20665 raft_consensus.cc:2454] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9145b72ee2554e6eaf55fec53d59a7fd in term 0.
I20260806 02:12:29.623989 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.617459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:36567 (local address 127.18.70.66:38305)
0806 02:12:29.618671 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.618683 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.618828 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.619175 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.619177 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.619204 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:29.619751 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.619761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.620735 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.620738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.622073 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.622082 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.623071 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.623075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.623194 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.623227 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.623763 (+ 536us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.623845 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1145,"mutex_wait_us":401,"thread_start_us":102,"threads_started":1}
I20260806 02:12:29.624212 20593 leader_election.cc:302] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45bf89be760448f39b7368267db96f98, 9145b72ee2554e6eaf55fec53d59a7fd; no voters:
I20260806 02:12:29.624212 20746 leader_election.cc:302] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45bf89be760448f39b7368267db96f98, 9145b72ee2554e6eaf55fec53d59a7fd; no voters:
I20260806 02:12:29.624305 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.617946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:36567 (local address 127.18.70.65:38293)
0806 02:12:29.618333 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.618347 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.618454 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.618929 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.618932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.618947 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:29.619041 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.619046 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.620043 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.620046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.621134 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.621142 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.622180 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.622184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.622499 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.622520 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.622582 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.622642 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":169,"thread_start_us":96,"threads_started":1}
I20260806 02:12:29.624526 21037 raft_consensus.cc:2781] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:29.624526 21039 raft_consensus.cc:2781] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:29.624614 21037 raft_consensus.cc:482] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:29.624614 21039 raft_consensus.cc:482] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:29.624670 21037 raft_consensus.cc:3037] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:29.624670 21039 raft_consensus.cc:3037] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:29.624919 20961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c2ede1858414906816c7a3ba3b1e887" candidate_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" is_pre_election: true
I20260806 02:12:29.625254 20961 raft_consensus.cc:2454] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9145b72ee2554e6eaf55fec53d59a7fd in term 0.
I20260806 02:12:29.626032 21039 raft_consensus.cc:504] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } }
I20260806 02:12:29.626031 21037 raft_consensus.cc:504] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } }
I20260806 02:12:29.626103 20960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c2ede1858414906816c7a3ba3b1e887" candidate_uuid: "45bf89be760448f39b7368267db96f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" is_pre_election: true
I20260806 02:12:29.626220 20960 raft_consensus.cc:2454] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45bf89be760448f39b7368267db96f98 in term 0.
I20260806 02:12:29.626385 21037 leader_election.cc:288] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [CANDIDATE]: Term 1 election: Requested vote from peers f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567), 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66:41165)
I20260806 02:12:29.626386 21039 leader_election.cc:288] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [CANDIDATE]: Term 1 election: Requested vote from peers 45bf89be760448f39b7368267db96f98 (127.18.70.65:34163), f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567)
I20260806 02:12:29.626793 20818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c2ede1858414906816c7a3ba3b1e887" candidate_uuid: "45bf89be760448f39b7368267db96f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9145b72ee2554e6eaf55fec53d59a7fd"
I20260806 02:12:29.626813 20665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c2ede1858414906816c7a3ba3b1e887" candidate_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45bf89be760448f39b7368267db96f98"
I20260806 02:12:29.626932 20960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c2ede1858414906816c7a3ba3b1e887" candidate_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42bbabc6cd44ff5b16e46849184a3e4"
I20260806 02:12:29.626973 20665 raft_consensus.cc:2379] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9145b72ee2554e6eaf55fec53d59a7fd in current term 1: Already voted for candidate 45bf89be760448f39b7368267db96f98 in this term.
I20260806 02:12:29.626967 20818 raft_consensus.cc:2379] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 45bf89be760448f39b7368267db96f98 in current term 1: Already voted for candidate 9145b72ee2554e6eaf55fec53d59a7fd in this term.
I20260806 02:12:29.627023 20960 raft_consensus.cc:3037] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:29.627404 20961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c2ede1858414906816c7a3ba3b1e887" candidate_uuid: "45bf89be760448f39b7368267db96f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42bbabc6cd44ff5b16e46849184a3e4"
I20260806 02:12:29.627887 20595 leader_election.cc:302] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45bf89be760448f39b7368267db96f98; no voters: 9145b72ee2554e6eaf55fec53d59a7fd, f42bbabc6cd44ff5b16e46849184a3e4
I20260806 02:12:29.628073 21037 raft_consensus.cc:2726] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 02:12:29.628409 20960 raft_consensus.cc:2454] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9145b72ee2554e6eaf55fec53d59a7fd in term 1.
I20260806 02:12:29.628697 20745 leader_election.cc:302] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9145b72ee2554e6eaf55fec53d59a7fd, f42bbabc6cd44ff5b16e46849184a3e4; no voters: 45bf89be760448f39b7368267db96f98
I20260806 02:12:29.628866 21039 raft_consensus.cc:2781] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:29.629086 21039 raft_consensus.cc:686] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 LEADER]: Becoming Leader. State: Replica: 9145b72ee2554e6eaf55fec53d59a7fd, State: Running, Role: LEADER
I20260806 02:12:29.629418 21039 consensus_queue.cc:237] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } }
I20260806 02:12:29.632613 20501 catalog_manager.cc:5495] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66). New cstate: current_term: 1 leader_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } health_report { overall_health: HEALTHY } } }
Aug 06 02:12:29 dist-test-slave-6k22 krb5kdc[20455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982348, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:29.673293 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.665163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42113 (local address 127.0.0.1:39856)
0806 02:12:29.665508 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.665519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.665648 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.665895 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.665897 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.666305 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:29.666400 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.666407 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.667139 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.667144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.667759 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.667769 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.668188 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.668191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.668265 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.668917 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:29.668929 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:29.670238 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:29.671685 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.671696 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.671703 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.671936 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.672229 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:29.672233 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:29.672235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:29.672315 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:29.672698 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:29.672706 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:29.672819 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.672930 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.673075 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":110,"threads_started":1}
I20260806 02:12:29.673293 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.665235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39856 (local address 127.18.70.126:42113)
0806 02:12:29.665373 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.665377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.665590 (+ 213us) server_negotiation.cc:407] Connection header received
0806 02:12:29.665738 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.665742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.665784 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.665871 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:29.666523 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.667026 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.667881 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.668085 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.668187 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.670377 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:29.670380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:29.670382 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:29.670401 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:29.671536 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.672042 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.672046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.672048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.672137 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:29.672411 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:29.672414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:29.672416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:29.672594 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:29.672698 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.672987 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.673126 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":46}
I20260806 02:12:29.677310 20501 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:39856:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 02:12:29.678049 20501 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:29.682880 20796 tablet_service.cc:1467] Processing CreateTablet for tablet d0e783e3df9e4195b9ba49fb4c2d5f10 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=423c64eb14c8466c8c3bcbe3a40a2fe5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 02:12:29.682854 20940 tablet_service.cc:1467] Processing CreateTablet for tablet d0e783e3df9e4195b9ba49fb4c2d5f10 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=423c64eb14c8466c8c3bcbe3a40a2fe5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 02:12:29.682853 20644 tablet_service.cc:1467] Processing CreateTablet for tablet d0e783e3df9e4195b9ba49fb4c2d5f10 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=423c64eb14c8466c8c3bcbe3a40a2fe5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 02:12:29.683559 20644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0e783e3df9e4195b9ba49fb4c2d5f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:29.683588 20940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0e783e3df9e4195b9ba49fb4c2d5f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:29.683598 20796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0e783e3df9e4195b9ba49fb4c2d5f10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:29.686013 21033 tablet_bootstrap.cc:492] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd: Bootstrap starting.
I20260806 02:12:29.686527 21032 tablet_bootstrap.cc:492] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98: Bootstrap starting.
I20260806 02:12:29.686728 21031 tablet_bootstrap.cc:492] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4: Bootstrap starting.
I20260806 02:12:29.687036 21033 tablet_bootstrap.cc:654] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:29.687507 21031 tablet_bootstrap.cc:654] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:29.687860 21032 tablet_bootstrap.cc:654] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:29.688679 21033 tablet_bootstrap.cc:492] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd: No bootstrap required, opened a new log
I20260806 02:12:29.688796 21033 ts_tablet_manager.cc:1397] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:12:29.688819 21031 tablet_bootstrap.cc:492] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4: No bootstrap required, opened a new log
I20260806 02:12:29.688894 21031 ts_tablet_manager.cc:1397] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:29.689134 21032 tablet_bootstrap.cc:492] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98: No bootstrap required, opened a new log
I20260806 02:12:29.689221 21032 ts_tablet_manager.cc:1397] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:12:29.689280 21033 raft_consensus.cc:348] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:29.689406 21033 raft_consensus.cc:374] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:29.689353 21031 raft_consensus.cc:348] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:29.689447 21033 raft_consensus.cc:729] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9145b72ee2554e6eaf55fec53d59a7fd, State: Initialized, Role: FOLLOWER
I20260806 02:12:29.689469 21031 raft_consensus.cc:374] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:29.689512 21031 raft_consensus.cc:729] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42bbabc6cd44ff5b16e46849184a3e4, State: Initialized, Role: FOLLOWER
I20260806 02:12:29.689565 21033 consensus_queue.cc:260] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:29.689652 21031 consensus_queue.cc:260] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:29.689679 21032 raft_consensus.cc:348] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:29.689787 21032 raft_consensus.cc:374] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:29.689822 21032 raft_consensus.cc:729] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45bf89be760448f39b7368267db96f98, State: Initialized, Role: FOLLOWER
I20260806 02:12:29.689889 21033 ts_tablet_manager.cc:1428] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:29.689850 21039 tablet_replica.cc:439] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } }
I20260806 02:12:29.689934 21031 ts_tablet_manager.cc:1428] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:29.689988 21039 tablet_replica.cc:442] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:29.690213 21038 tablet_replica.cc:439] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } }
I20260806 02:12:29.690387 21038 tablet_replica.cc:442] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:29.689971 21032 consensus_queue.cc:260] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:29.690811 21037 tablet_replica.cc:439] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } }
I20260806 02:12:29.690950 21037 tablet_replica.cc:442] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:29.691514 21032 ts_tablet_manager.cc:1428] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260806 02:12:29.749378 21008 tablet.cc:2369] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 02:12:29.761044 20865 tablet.cc:2369] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 02:12:29.787964 20718 tablet.cc:2369] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:12:29.902877 21037 raft_consensus.cc:482] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:29.903052 21037 raft_consensus.cc:504] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:29.903640 21037 leader_election.cc:288] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66:41165), f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567)
I20260806 02:12:29.904109 20960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0e783e3df9e4195b9ba49fb4c2d5f10" candidate_uuid: "45bf89be760448f39b7368267db96f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" is_pre_election: true
I20260806 02:12:29.904110 20818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0e783e3df9e4195b9ba49fb4c2d5f10" candidate_uuid: "45bf89be760448f39b7368267db96f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" is_pre_election: true
I20260806 02:12:29.904258 20960 raft_consensus.cc:2454] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45bf89be760448f39b7368267db96f98 in term 0.
I20260806 02:12:29.904309 20818 raft_consensus.cc:2454] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45bf89be760448f39b7368267db96f98 in term 0.
I20260806 02:12:29.904644 20593 leader_election.cc:302] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45bf89be760448f39b7368267db96f98, 9145b72ee2554e6eaf55fec53d59a7fd; no voters:
I20260806 02:12:29.904829 21037 raft_consensus.cc:2781] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:29.904894 21037 raft_consensus.cc:482] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:29.904934 21037 raft_consensus.cc:3037] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:29.905925 21037 raft_consensus.cc:504] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:29.906298 21037 leader_election.cc:288] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [CANDIDATE]: Term 1 election: Requested vote from peers 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66:41165), f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567)
I20260806 02:12:29.906672 20818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0e783e3df9e4195b9ba49fb4c2d5f10" candidate_uuid: "45bf89be760448f39b7368267db96f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9145b72ee2554e6eaf55fec53d59a7fd"
I20260806 02:12:29.906781 20818 raft_consensus.cc:3037] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:29.906803 20960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0e783e3df9e4195b9ba49fb4c2d5f10" candidate_uuid: "45bf89be760448f39b7368267db96f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42bbabc6cd44ff5b16e46849184a3e4"
I20260806 02:12:29.906932 20960 raft_consensus.cc:3037] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:29.907840 20818 raft_consensus.cc:2454] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45bf89be760448f39b7368267db96f98 in term 1.
I20260806 02:12:29.908001 20960 raft_consensus.cc:2454] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45bf89be760448f39b7368267db96f98 in term 1.
I20260806 02:12:29.908133 20593 leader_election.cc:302] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45bf89be760448f39b7368267db96f98, 9145b72ee2554e6eaf55fec53d59a7fd; no voters:
I20260806 02:12:29.908527 21037 raft_consensus.cc:2781] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:29.908784 21037 raft_consensus.cc:686] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [term 1 LEADER]: Becoming Leader. State: Replica: 45bf89be760448f39b7368267db96f98, State: Running, Role: LEADER
I20260806 02:12:29.909145 21037 consensus_queue.cc:237] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:29.911379 21055 tablet_replica.cc:439] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98: TxnStatusTablet state changed. Reason: New leader 45bf89be760448f39b7368267db96f98. Latest consensus state: current_term: 1 leader_uuid: "45bf89be760448f39b7368267db96f98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } }
I20260806 02:12:29.911495 21055 tablet_replica.cc:442] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98: This TxnStatusTablet replica's current role is: LEADER
I20260806 02:12:29.911798 21057 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 02:12:29.912048 21057 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 02:12:29.912539 20502 catalog_manager.cc:5495] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45bf89be760448f39b7368267db96f98 (127.18.70.65). New cstate: current_term: 1 leader_uuid: "45bf89be760448f39b7368267db96f98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } health_report { overall_health: UNKNOWN } } }
I20260806 02:12:29.933110 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.928735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51298 (local address 127.18.70.65:34163)
0806 02:12:29.928894 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.928899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.928934 (+ 35us) server_negotiation.cc:407] Connection header received
0806 02:12:29.929111 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.929114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.929175 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.929285 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:29.929495 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.930248 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.931541 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.931818 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.931931 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.932315 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:29.932740 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:29.932839 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.932965 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.933012 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 02:12:29.933163 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.928605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:34163 (local address 127.0.0.1:51298)
0806 02:12:29.928879 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.928893 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.928984 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.929303 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.929305 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.929310 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 02:12:29.929395 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.929399 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.930404 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.930408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.931379 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.931391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.931964 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.931967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.932067 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.932119 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 02:12:29.932852 (+ 733us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 02:12:29.932868 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.932945 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.933027 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260806 02:12:29.938200 20818 raft_consensus.cc:1261] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Refusing update from remote peer 45bf89be760448f39b7368267db96f98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:29.938279 20960 raft_consensus.cc:1261] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Refusing update from remote peer 45bf89be760448f39b7368267db96f98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:29.938786 21055 consensus_queue.cc:1035] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:29.939035 21037 consensus_queue.cc:1035] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:29.942055 21039 tablet_replica.cc:439] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd: TxnStatusTablet state changed. Reason: New leader 45bf89be760448f39b7368267db96f98. Latest consensus state: current_term: 1 leader_uuid: "45bf89be760448f39b7368267db96f98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } }
I20260806 02:12:29.942162 21039 tablet_replica.cc:442] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:29.943182 21038 tablet_replica.cc:439] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4: TxnStatusTablet state changed. Reason: New leader 45bf89be760448f39b7368267db96f98. Latest consensus state: current_term: 1 leader_uuid: "45bf89be760448f39b7368267db96f98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } }
I20260806 02:12:29.943280 21038 tablet_replica.cc:442] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:29.946630 21055 tablet_replica.cc:439] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45bf89be760448f39b7368267db96f98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } }
I20260806 02:12:29.946750 21055 tablet_replica.cc:442] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98: This TxnStatusTablet replica's current role is: LEADER
I20260806 02:12:29.947141 21038 tablet_replica.cc:439] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45bf89be760448f39b7368267db96f98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } }
I20260806 02:12:29.947252 21038 tablet_replica.cc:442] T d0e783e3df9e4195b9ba49fb4c2d5f10 P f42bbabc6cd44ff5b16e46849184a3e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:29.947794 21062 mvcc.cc:205] Tried to move back new op lower bound from 7315383705340567552 to 7315383705230704640. Current Snapshot: MvccSnapshot[applied={T|T < 7315383705340567552}]
I20260806 02:12:29.948968 21055 tablet_replica.cc:439] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45bf89be760448f39b7368267db96f98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } }
I20260806 02:12:29.949067 21055 tablet_replica.cc:442] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 45bf89be760448f39b7368267db96f98: This TxnStatusTablet replica's current role is: LEADER
I20260806 02:12:29.950158 21039 tablet_replica.cc:439] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45bf89be760448f39b7368267db96f98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } } }
I20260806 02:12:29.950264 21039 tablet_replica.cc:442] T d0e783e3df9e4195b9ba49fb4c2d5f10 P 9145b72ee2554e6eaf55fec53d59a7fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:29.951107 21063 mvcc.cc:205] Tried to move back new op lower bound from 7315383705340567552 to 7315383705230704640. Current Snapshot: MvccSnapshot[applied={T|T < 7315383705340567552}]
I20260806 02:12:29.962500 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.958676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43984 (local address 127.18.70.66:41165)
0806 02:12:29.958808 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.958813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.958829 (+ 16us) server_negotiation.cc:407] Connection header received
0806 02:12:29.958982 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.958985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.959045 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.959153 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:29.959381 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.960174 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.960998 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.961343 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.961462 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.961699 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:29.962158 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:29.962281 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.962319 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.962373 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 02:12:29.973939 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.969813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:45323 (local address 127.18.70.65:34163)
0806 02:12:29.969948 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:29.969952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:29.970104 (+ 152us) server_negotiation.cc:407] Connection header received
0806 02:12:29.970264 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:29.970267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:29.970316 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:29.970395 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:29.970738 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.971465 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.972731 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.973066 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.973174 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.973496 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:29.973573 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:29.973646 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:29.973759 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 02:12:29.973811 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 02:12:29.973965 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:29.969705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:34163 (local address 127.18.70.66:45323)
0806 02:12:29.970017 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:29.970029 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:29.970147 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:29.970406 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:29.970409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:29.970417 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 02:12:29.970607 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.970617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.971594 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.971598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.972606 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:29.972615 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:29.973184 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:29.973188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:29.973292 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:29.973343 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 02:12:29.973659 (+ 316us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 02:12:29.973665 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 02:12:29.973724 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 02:12:29.973808 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":97,"threads_started":1}
I20260806 02:12:29.981943 20960 raft_consensus.cc:1261] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Refusing update from remote peer 9145b72ee2554e6eaf55fec53d59a7fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:29.982072 20665 raft_consensus.cc:1261] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Refusing update from remote peer 9145b72ee2554e6eaf55fec53d59a7fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:29.982359 21044 consensus_queue.cc:1035] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:29.982609 21039 consensus_queue.cc:1035] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:29.994673 21061 mvcc.cc:205] Tried to move back new op lower bound from 7315383705522073600 to 7315383704084660224. Current Snapshot: MvccSnapshot[applied={T|T < 7315383705522073600}]
I20260806 02:12:30.007779 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:30.003715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:41527 (local address 127.18.70.66:41165)
0806 02:12:30.003847 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:30.003852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:30.003868 (+ 16us) server_negotiation.cc:407] Connection header received
0806 02:12:30.003923 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:30.003926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:30.003967 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:30.004037 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:30.004317 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.004897 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.006736 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.006941 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.007023 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:30.007335 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:30.007420 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:30.007474 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:30.007617 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 02:12:30.007666 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 02:12:30.007848 21079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:30.003285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:41165 (local address 127.18.70.65:41527)
0806 02:12:30.003680 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:30.003697 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:30.003793 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:30.004068 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:30.004072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:30.004080 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 02:12:30.004223 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:30.004229 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.005166 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.005173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:30.006627 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:30.006640 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.007049 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.007053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:30.007148 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:30.007202 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 02:12:30.007505 (+ 303us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 02:12:30.007512 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 02:12:30.007599 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 02:12:30.007679 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":152,"threads_started":1}
I20260806 02:12:30.035270 20502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:30.035470 20502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:30.037912 20502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d: Sending DeleteTablet for 3 replicas of tablet 7c2ede1858414906816c7a3ba3b1e887
I20260806 02:12:30.042682 20940 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c2ede1858414906816c7a3ba3b1e887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:30 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:42598
I20260806 02:12:30.043164 21082 tablet_replica.cc:330] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4: stopping tablet replica
I20260806 02:12:30.043371 20796 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c2ede1858414906816c7a3ba3b1e887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:30 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:43976
I20260806 02:12:30.043722 21082 raft_consensus.cc:2229] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:30.043823 20644 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c2ede1858414906816c7a3ba3b1e887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:30 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:51296
I20260806 02:12:30.044083 21082 raft_consensus.cc:2258] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:30.044971 20502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:12:30.045390 20502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:30.045830 21082 ts_tablet_manager.cc:1905] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:30.050380 20796 tablet_service.cc:1467] Processing CreateTablet for tablet cee5a8ea93e847368e8a7c75371031a1 (DEFAULT_TABLE table=test-table [id=50787128e95a46ab8ae94cd47d95327d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:30.050637 20796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cee5a8ea93e847368e8a7c75371031a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:30.053284 21033 tablet_bootstrap.cc:492] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd: Bootstrap starting.
I20260806 02:12:30.054203 21033 tablet_bootstrap.cc:654] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:30.054234 21084 tablet_replica.cc:330] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd: stopping tablet replica
I20260806 02:12:30.054551 21084 raft_consensus.cc:2229] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:12:30.054883 21084 raft_consensus.cc:2258] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:30.055234 21085 tablet_replica.cc:330] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98: stopping tablet replica
I20260806 02:12:30.055804 20940 tablet_service.cc:1467] Processing CreateTablet for tablet cee5a8ea93e847368e8a7c75371031a1 (DEFAULT_TABLE table=test-table [id=50787128e95a46ab8ae94cd47d95327d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:30.055789 20644 tablet_service.cc:1467] Processing CreateTablet for tablet cee5a8ea93e847368e8a7c75371031a1 (DEFAULT_TABLE table=test-table [id=50787128e95a46ab8ae94cd47d95327d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:30.056041 20644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cee5a8ea93e847368e8a7c75371031a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:30.056398 21085 raft_consensus.cc:2229] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:30.057024 21085 raft_consensus.cc:2258] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:30.059068 21032 tablet_bootstrap.cc:492] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98: Bootstrap starting.
I20260806 02:12:30.060140 21032 tablet_bootstrap.cc:654] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:30.061399 20940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cee5a8ea93e847368e8a7c75371031a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:30.063812 21031 tablet_bootstrap.cc:492] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4: Bootstrap starting.
I20260806 02:12:30.064186 21082 ts_tablet_manager.cc:1918] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:12:30.064358 21082 log.cc:1196] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/7c2ede1858414906816c7a3ba3b1e887
I20260806 02:12:30.064756 21031 tablet_bootstrap.cc:654] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:30.065485 21082 ts_tablet_manager.cc:1939] T 7c2ede1858414906816c7a3ba3b1e887 P f42bbabc6cd44ff5b16e46849184a3e4: Deleting consensus metadata
I20260806 02:12:30.066682 21084 ts_tablet_manager.cc:1905] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:30.066820 20487 catalog_manager.cc:4841] TS f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567): tablet 7c2ede1858414906816c7a3ba3b1e887 (table test-table [id=e4c09fc559a046fdb13aa674472e09d7]) successfully deleted
I20260806 02:12:30.067910 21031 tablet_bootstrap.cc:492] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4: No bootstrap required, opened a new log
I20260806 02:12:30.068013 21031 ts_tablet_manager.cc:1397] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 02:12:30.068468 21032 tablet_bootstrap.cc:492] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98: No bootstrap required, opened a new log
I20260806 02:12:30.068557 21032 ts_tablet_manager.cc:1397] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20260806 02:12:30.068495 21031 raft_consensus.cc:348] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:30.068632 21031 raft_consensus.cc:374] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:30.068722 21031 raft_consensus.cc:729] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42bbabc6cd44ff5b16e46849184a3e4, State: Initialized, Role: FOLLOWER
I20260806 02:12:30.068845 21031 consensus_queue.cc:260] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:30.069001 21032 raft_consensus.cc:348] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:30.069093 21031 ts_tablet_manager.cc:1428] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:30.069128 21032 raft_consensus.cc:374] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:30.069178 21032 raft_consensus.cc:729] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45bf89be760448f39b7368267db96f98, State: Initialized, Role: FOLLOWER
I20260806 02:12:30.069207 21085 ts_tablet_manager.cc:1905] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:30.069306 21032 consensus_queue.cc:260] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:30.069589 21032 ts_tablet_manager.cc:1428] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 02:12:30.075467 21033 tablet_bootstrap.cc:492] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd: No bootstrap required, opened a new log
I20260806 02:12:30.075613 21033 ts_tablet_manager.cc:1397] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20260806 02:12:30.076108 21033 raft_consensus.cc:348] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:30.076237 21033 raft_consensus.cc:374] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:30.076279 21033 raft_consensus.cc:729] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9145b72ee2554e6eaf55fec53d59a7fd, State: Initialized, Role: FOLLOWER
I20260806 02:12:30.076409 21033 consensus_queue.cc:260] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:30.076702 21033 ts_tablet_manager.cc:1428] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 02:12:30.079222 21084 ts_tablet_manager.cc:1918] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:12:30.079325 21084 log.cc:1196] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/7c2ede1858414906816c7a3ba3b1e887
I20260806 02:12:30.079739 21084 ts_tablet_manager.cc:1939] T 7c2ede1858414906816c7a3ba3b1e887 P 9145b72ee2554e6eaf55fec53d59a7fd: Deleting consensus metadata
I20260806 02:12:30.081037 20485 catalog_manager.cc:4841] TS 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66:41165): tablet 7c2ede1858414906816c7a3ba3b1e887 (table test-table [id=e4c09fc559a046fdb13aa674472e09d7]) successfully deleted
I20260806 02:12:30.082306 21085 ts_tablet_manager.cc:1918] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:12:30.082438 21085 log.cc:1196] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/wal/wals/7c2ede1858414906816c7a3ba3b1e887
I20260806 02:12:30.082737 21085 ts_tablet_manager.cc:1939] T 7c2ede1858414906816c7a3ba3b1e887 P 45bf89be760448f39b7368267db96f98: Deleting consensus metadata
I20260806 02:12:30.083849 20488 catalog_manager.cc:4841] TS 45bf89be760448f39b7368267db96f98 (127.18.70.65:34163): tablet 7c2ede1858414906816c7a3ba3b1e887 (table test-table [id=e4c09fc559a046fdb13aa674472e09d7]) successfully deleted
I20260806 02:12:30.389351 21038 raft_consensus.cc:482] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:30.389519 21038 raft_consensus.cc:504] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:30.390689 21038 leader_election.cc:288] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45bf89be760448f39b7368267db96f98 (127.18.70.65:34163), 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66:41165)
I20260806 02:12:30.394810 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:30.390902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:34163 (local address 127.18.70.67:33427)
0806 02:12:30.391171 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:30.391184 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:30.391338 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:30.391706 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:30.391709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:30.391727 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:30.391863 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:30.391871 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.392678 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.392682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:30.393589 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:30.393598 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.394360 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.394364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:30.394477 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:30.394502 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 02:12:30.394574 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 02:12:30.394630 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":50,"thread_start_us":82,"threads_started":1}
I20260806 02:12:30.394862 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:30.391031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:33427 (local address 127.18.70.65:34163)
0806 02:12:30.391198 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:30.391204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:30.391294 (+ 90us) server_negotiation.cc:407] Connection header received
0806 02:12:30.391500 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:30.391504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:30.391597 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:30.391710 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:30.391981 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.392560 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.393729 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.394233 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.394363 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:30.394430 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:30.394619 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 02:12:30.394720 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 02:12:30.394922 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:30.391325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:58569 (local address 127.18.70.66:41165)
0806 02:12:30.391486 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:30.391489 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:30.391502 (+ 13us) server_negotiation.cc:407] Connection header received
0806 02:12:30.391682 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:30.391686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:30.391735 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:30.391814 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:30.392105 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.392577 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.393808 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.394257 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.394394 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:30.394451 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:30.394724 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 02:12:30.394796 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 02:12:30.394980 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:30.391027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:41165 (local address 127.18.70.67:58569)
0806 02:12:30.391372 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:30.391388 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:30.391508 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:30.391863 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:30.391865 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:30.391881 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:30.391988 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:30.391994 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.392730 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.392734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:30.393641 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:30.393682 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.394419 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.394423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:30.394550 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:30.394602 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 02:12:30.394689 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 02:12:30.394758 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":28,"thread_start_us":91,"threads_started":1}
I20260806 02:12:30.395387 20665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cee5a8ea93e847368e8a7c75371031a1" candidate_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45bf89be760448f39b7368267db96f98" is_pre_election: true
I20260806 02:12:30.395561 20665 raft_consensus.cc:2454] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f42bbabc6cd44ff5b16e46849184a3e4 in term 0.
I20260806 02:12:30.395612 20818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cee5a8ea93e847368e8a7c75371031a1" candidate_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" is_pre_election: true
I20260806 02:12:30.395780 20818 raft_consensus.cc:2454] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f42bbabc6cd44ff5b16e46849184a3e4 in term 0.
I20260806 02:12:30.395983 20893 leader_election.cc:302] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45bf89be760448f39b7368267db96f98, f42bbabc6cd44ff5b16e46849184a3e4; no voters:
I20260806 02:12:30.396226 21038 raft_consensus.cc:2781] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:30.396287 21038 raft_consensus.cc:482] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:30.396327 21038 raft_consensus.cc:3037] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:30.397295 21038 raft_consensus.cc:504] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:30.397627 21038 leader_election.cc:288] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [CANDIDATE]: Term 1 election: Requested vote from peers 45bf89be760448f39b7368267db96f98 (127.18.70.65:34163), 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66:41165)
I20260806 02:12:30.397933 20665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cee5a8ea93e847368e8a7c75371031a1" candidate_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45bf89be760448f39b7368267db96f98"
I20260806 02:12:30.398000 20818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cee5a8ea93e847368e8a7c75371031a1" candidate_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9145b72ee2554e6eaf55fec53d59a7fd"
I20260806 02:12:30.398128 20818 raft_consensus.cc:3037] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:30.398128 20665 raft_consensus.cc:3037] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:30.399099 20818 raft_consensus.cc:2454] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f42bbabc6cd44ff5b16e46849184a3e4 in term 1.
I20260806 02:12:30.399324 20665 raft_consensus.cc:2454] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f42bbabc6cd44ff5b16e46849184a3e4 in term 1.
I20260806 02:12:30.399441 20890 leader_election.cc:302] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9145b72ee2554e6eaf55fec53d59a7fd, f42bbabc6cd44ff5b16e46849184a3e4; no voters:
I20260806 02:12:30.399698 21038 raft_consensus.cc:2781] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:30.399920 21038 raft_consensus.cc:686] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 LEADER]: Becoming Leader. State: Replica: f42bbabc6cd44ff5b16e46849184a3e4, State: Running, Role: LEADER
I20260806 02:12:30.400204 21038 consensus_queue.cc:237] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:30.402726 20501 catalog_manager.cc:5495] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67). New cstate: current_term: 1 leader_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } health_report { overall_health: HEALTHY } } }
I20260806 02:12:30.440758 21093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:30.437317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.18.70.67:36567)
0806 02:12:30.437580 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:30.437586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:30.437597 (+ 11us) server_negotiation.cc:407] Connection header received
0806 02:12:30.437661 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:30.437663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:30.437709 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:30.437815 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:30.438021 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.438559 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.439301 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.439524 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.439690 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:30.439902 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:30.440366 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:30.440464 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:30.440520 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 02:12:30.440562 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":119,"threads_started":1}
I20260806 02:12:30.450834 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:30.447373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:57013 (local address 127.18.70.65:34163)
0806 02:12:30.447504 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:30.447507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:30.447719 (+ 212us) server_negotiation.cc:407] Connection header received
0806 02:12:30.447804 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:30.447806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:30.447846 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:30.447906 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:30.448480 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.449005 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.449923 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.450129 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.450201 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:30.450456 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:30.450515 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:30.450567 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:30.450667 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 02:12:30.450706 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 02:12:30.450875 21096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:30.447319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:34163 (local address 127.18.70.67:57013)
0806 02:12:30.447637 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:30.447649 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:30.447729 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:30.447933 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:30.447935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:30.447940 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 02:12:30.448372 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:30.448380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.449097 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.449099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:30.449799 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:30.449808 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.450236 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.450239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:30.450300 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:30.450340 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 02:12:30.450590 (+ 250us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 02:12:30.450595 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 02:12:30.450660 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 02:12:30.450719 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":106,"threads_started":1}
I20260806 02:12:30.456457 20665 raft_consensus.cc:1261] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Refusing update from remote peer f42bbabc6cd44ff5b16e46849184a3e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:30.456456 20818 raft_consensus.cc:1261] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Refusing update from remote peer f42bbabc6cd44ff5b16e46849184a3e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:30.456944 21091 consensus_queue.cc:1035] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:30.457181 21038 consensus_queue.cc:1035] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:30.474833 21093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:30.471797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:50397 (local address 127.18.70.67:36567)
0806 02:12:30.471894 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:30.471898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:30.471917 (+ 19us) server_negotiation.cc:407] Connection header received
0806 02:12:30.471953 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:30.471954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:30.471990 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:30.472047 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:30.472288 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.472768 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.473690 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.473991 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.474088 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:30.474330 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:30.474451 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:30.474522 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:30.474621 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 02:12:30.474678 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 02:12:30.474843 21079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:30.471539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:36567 (local address 127.18.70.65:50397)
0806 02:12:30.471701 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:30.471718 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:30.471798 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:30.472098 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:30.472100 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:30.472108 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 02:12:30.472192 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:30.472196 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.472854 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.472856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:30.473560 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:30.473571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:30.474082 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:30.474086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:30.474178 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:30.474219 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 02:12:30.474530 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 02:12:30.474539 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 02:12:30.474613 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 02:12:30.474732 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 02:12:30.475529 21062 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 02:12:32.031386 20502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:32.031566 20502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:32.033806 20502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d: Sending DeleteTablet for 3 replicas of tablet cee5a8ea93e847368e8a7c75371031a1
I20260806 02:12:32.034350 20644 tablet_service.cc:1514] Processing DeleteTablet for tablet cee5a8ea93e847368e8a7c75371031a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:32 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:51296
I20260806 02:12:32.034363 20796 tablet_service.cc:1514] Processing DeleteTablet for tablet cee5a8ea93e847368e8a7c75371031a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:32 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:43976
I20260806 02:12:32.034475 20940 tablet_service.cc:1514] Processing DeleteTablet for tablet cee5a8ea93e847368e8a7c75371031a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:32 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:42598
I20260806 02:12:32.034752 21124 tablet_replica.cc:330] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4: stopping tablet replica
I20260806 02:12:32.034876 21124 raft_consensus.cc:2229] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:12:32.035178 21124 raft_consensus.cc:2258] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:32.035480 21122 tablet_replica.cc:330] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98: stopping tablet replica
I20260806 02:12:32.035652 21122 raft_consensus.cc:2229] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:32.035792 21122 raft_consensus.cc:2258] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:32.035950 21124 ts_tablet_manager.cc:1905] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:32.036152 21123 tablet_replica.cc:330] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd: stopping tablet replica
I20260806 02:12:32.036298 21123 raft_consensus.cc:2229] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:32.036412 21123 raft_consensus.cc:2258] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:32.036679 21122 ts_tablet_manager.cc:1905] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:32.038573 21123 ts_tablet_manager.cc:1905] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:32.038830 21124 ts_tablet_manager.cc:1918] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 02:12:32.038908 21124 log.cc:1196] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/cee5a8ea93e847368e8a7c75371031a1
I20260806 02:12:32.039253 21124 ts_tablet_manager.cc:1939] T cee5a8ea93e847368e8a7c75371031a1 P f42bbabc6cd44ff5b16e46849184a3e4: Deleting consensus metadata
I20260806 02:12:32.039884 20487 catalog_manager.cc:4841] TS f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567): tablet cee5a8ea93e847368e8a7c75371031a1 (table test-table [id=50787128e95a46ab8ae94cd47d95327d]) successfully deleted
I20260806 02:12:32.040060 21122 ts_tablet_manager.cc:1918] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 02:12:32.040148 21122 log.cc:1196] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/wal/wals/cee5a8ea93e847368e8a7c75371031a1
I20260806 02:12:32.040449 21122 ts_tablet_manager.cc:1939] T cee5a8ea93e847368e8a7c75371031a1 P 45bf89be760448f39b7368267db96f98: Deleting consensus metadata
I20260806 02:12:32.041079 20488 catalog_manager.cc:4841] TS 45bf89be760448f39b7368267db96f98 (127.18.70.65:34163): tablet cee5a8ea93e847368e8a7c75371031a1 (table test-table [id=50787128e95a46ab8ae94cd47d95327d]) successfully deleted
I20260806 02:12:32.041270 21123 ts_tablet_manager.cc:1918] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 02:12:32.041343 21123 log.cc:1196] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/cee5a8ea93e847368e8a7c75371031a1
I20260806 02:12:32.041631 21123 ts_tablet_manager.cc:1939] T cee5a8ea93e847368e8a7c75371031a1 P 9145b72ee2554e6eaf55fec53d59a7fd: Deleting consensus metadata
I20260806 02:12:32.042217 20485 catalog_manager.cc:4841] TS 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66:41165): tablet cee5a8ea93e847368e8a7c75371031a1 (table test-table [id=50787128e95a46ab8ae94cd47d95327d]) successfully deleted
Aug 06 02:12:32 dist-test-slave-6k22 krb5kdc[20455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982352, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 02:12:32 dist-test-slave-6k22 krb5kdc[20455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982352, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:32.077721 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:32.062936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39872 (local address 127.18.70.126:42113)
0806 02:12:32.063184 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:32.063190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:32.063203 (+ 13us) server_negotiation.cc:407] Connection header received
0806 02:12:32.063333 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:32.063336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:32.063379 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:32.063459 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:32.064215 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:32.064925 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:32.065836 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:32.066097 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:32.066194 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:32.074184 (+ 7990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:32.074188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:32.074191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:32.074217 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:32.075824 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:32.076412 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:32.076415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:32.076417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:32.076500 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:32.076796 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:32.076800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:32.076801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:32.077022 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:32.077136 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:32.077360 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 02:12:32.077490 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":106,"threads_started":1}
I20260806 02:12:32.080549 20502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:12:32.081086 20502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:32.086001 20644 tablet_service.cc:1467] Processing CreateTablet for tablet caa1a8173b6643aaa5dc19d324830dae (DEFAULT_TABLE table=test-table [id=9fad1f5122334ba9be03e1a72ad71ff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:32.086045 20796 tablet_service.cc:1467] Processing CreateTablet for tablet caa1a8173b6643aaa5dc19d324830dae (DEFAULT_TABLE table=test-table [id=9fad1f5122334ba9be03e1a72ad71ff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:32.086308 20644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa1a8173b6643aaa5dc19d324830dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:32.086308 20796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa1a8173b6643aaa5dc19d324830dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:32.086238 20940 tablet_service.cc:1467] Processing CreateTablet for tablet caa1a8173b6643aaa5dc19d324830dae (DEFAULT_TABLE table=test-table [id=9fad1f5122334ba9be03e1a72ad71ff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:32.086567 20940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa1a8173b6643aaa5dc19d324830dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:32.089059 21135 tablet_bootstrap.cc:492] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4: Bootstrap starting.
I20260806 02:12:32.089250 21134 tablet_bootstrap.cc:492] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98: Bootstrap starting.
I20260806 02:12:32.089989 21135 tablet_bootstrap.cc:654] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:32.090075 21134 tablet_bootstrap.cc:654] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:32.090250 21133 tablet_bootstrap.cc:492] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd: Bootstrap starting.
I20260806 02:12:32.091216 21135 tablet_bootstrap.cc:492] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4: No bootstrap required, opened a new log
I20260806 02:12:32.091228 21134 tablet_bootstrap.cc:492] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98: No bootstrap required, opened a new log
I20260806 02:12:32.091238 21133 tablet_bootstrap.cc:654] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:32.091336 21135 ts_tablet_manager.cc:1397] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 02:12:32.091336 21134 ts_tablet_manager.cc:1397] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:32.091785 21134 raft_consensus.cc:348] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.091785 21135 raft_consensus.cc:348] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.091902 21135 raft_consensus.cc:374] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:32.091902 21134 raft_consensus.cc:374] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:32.091938 21134 raft_consensus.cc:729] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45bf89be760448f39b7368267db96f98, State: Initialized, Role: FOLLOWER
I20260806 02:12:32.091938 21135 raft_consensus.cc:729] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42bbabc6cd44ff5b16e46849184a3e4, State: Initialized, Role: FOLLOWER
I20260806 02:12:32.092063 21135 consensus_queue.cc:260] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.092087 21134 consensus_queue.cc:260] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.092360 21135 ts_tablet_manager.cc:1428] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:32.092851 21134 ts_tablet_manager.cc:1428] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:32.093169 21133 tablet_bootstrap.cc:492] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd: No bootstrap required, opened a new log
I20260806 02:12:32.093267 21133 ts_tablet_manager.cc:1397] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 02:12:32.093685 21133 raft_consensus.cc:348] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.093812 21133 raft_consensus.cc:374] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:32.093860 21133 raft_consensus.cc:729] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9145b72ee2554e6eaf55fec53d59a7fd, State: Initialized, Role: FOLLOWER
I20260806 02:12:32.094017 21133 consensus_queue.cc:260] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.094413 21133 ts_tablet_manager.cc:1428] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260806 02:12:32.131835 21108 raft_consensus.cc:482] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:32.131950 21108 raft_consensus.cc:504] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.132414 21108 leader_election.cc:288] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66:41165), f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567)
I20260806 02:12:32.132782 20818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa1a8173b6643aaa5dc19d324830dae" candidate_uuid: "45bf89be760448f39b7368267db96f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" is_pre_election: true
I20260806 02:12:32.132791 20960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa1a8173b6643aaa5dc19d324830dae" candidate_uuid: "45bf89be760448f39b7368267db96f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" is_pre_election: true
I20260806 02:12:32.132925 20818 raft_consensus.cc:2454] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45bf89be760448f39b7368267db96f98 in term 0.
I20260806 02:12:32.132925 20960 raft_consensus.cc:2454] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45bf89be760448f39b7368267db96f98 in term 0.
I20260806 02:12:32.133239 20595 leader_election.cc:302] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45bf89be760448f39b7368267db96f98, f42bbabc6cd44ff5b16e46849184a3e4; no voters:
I20260806 02:12:32.133457 21108 raft_consensus.cc:2781] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:32.133544 21108 raft_consensus.cc:482] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:32.133599 21108 raft_consensus.cc:3037] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:32.134636 21108 raft_consensus.cc:504] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.135011 21108 leader_election.cc:288] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [CANDIDATE]: Term 1 election: Requested vote from peers 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66:41165), f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567)
I20260806 02:12:32.135350 20818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa1a8173b6643aaa5dc19d324830dae" candidate_uuid: "45bf89be760448f39b7368267db96f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9145b72ee2554e6eaf55fec53d59a7fd"
I20260806 02:12:32.135350 20960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa1a8173b6643aaa5dc19d324830dae" candidate_uuid: "45bf89be760448f39b7368267db96f98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42bbabc6cd44ff5b16e46849184a3e4"
I20260806 02:12:32.135478 20960 raft_consensus.cc:3037] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:32.135478 20818 raft_consensus.cc:3037] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:32.136621 20818 raft_consensus.cc:2454] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45bf89be760448f39b7368267db96f98 in term 1.
I20260806 02:12:32.136621 20960 raft_consensus.cc:2454] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45bf89be760448f39b7368267db96f98 in term 1.
I20260806 02:12:32.136907 20595 leader_election.cc:302] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45bf89be760448f39b7368267db96f98, f42bbabc6cd44ff5b16e46849184a3e4; no voters:
I20260806 02:12:32.137110 21108 raft_consensus.cc:2781] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:32.137205 21108 raft_consensus.cc:686] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [term 1 LEADER]: Becoming Leader. State: Replica: 45bf89be760448f39b7368267db96f98, State: Running, Role: LEADER
I20260806 02:12:32.137339 21108 consensus_queue.cc:237] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.138913 20502 catalog_manager.cc:5495] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45bf89be760448f39b7368267db96f98 (127.18.70.65). New cstate: current_term: 1 leader_uuid: "45bf89be760448f39b7368267db96f98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } health_report { overall_health: UNKNOWN } } }
I20260806 02:12:32.160182 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:32.157191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51312 (local address 127.18.70.65:34163)
0806 02:12:32.157443 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:32.157450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:32.157476 (+ 26us) server_negotiation.cc:407] Connection header received
0806 02:12:32.157523 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:32.157525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:32.157564 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:32.157625 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:32.157840 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:32.158335 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:32.159148 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:32.159370 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:32.159444 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:32.159727 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:32.159797 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:32.159853 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:32.159952 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 02:12:32.159998 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":110,"threads_started":1}
I20260806 02:12:32.168872 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:32.165800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:57969 (local address 127.18.70.65:34163)
0806 02:12:32.165914 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:32.165918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:32.166010 (+ 92us) server_negotiation.cc:407] Connection header received
0806 02:12:32.166150 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:32.166153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:32.166193 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:32.166269 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:32.166492 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:32.166948 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:32.167884 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:32.168085 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:32.168187 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:32.168444 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:32.168502 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:32.168574 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:32.168659 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 02:12:32.168742 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":45}
I20260806 02:12:32.168874 21145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:32.165634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:34163 (local address 127.18.70.65:57969)
0806 02:12:32.165916 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:32.165931 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:32.166051 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:32.166277 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:32.166280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:32.166287 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 02:12:32.166374 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:32.166379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:32.167048 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:32.167053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:32.167754 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:32.167763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:32.168188 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:32.168191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:32.168298 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:32.168327 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 02:12:32.168588 (+ 261us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 02:12:32.168594 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 02:12:32.168651 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 02:12:32.168762 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":21,"thread_start_us":123,"threads_started":1}
I20260806 02:12:32.174356 20818 raft_consensus.cc:1261] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Refusing update from remote peer 45bf89be760448f39b7368267db96f98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:32.182307 20960 raft_consensus.cc:1261] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Refusing update from remote peer 45bf89be760448f39b7368267db96f98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:32.182463 21108 consensus_queue.cc:1035] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:32.182819 21103 consensus_queue.cc:1035] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:32.216447 20501 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:32.216612 20501 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:32.218716 20501 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d: Sending DeleteTablet for 3 replicas of tablet caa1a8173b6643aaa5dc19d324830dae
I20260806 02:12:32.219287 20644 tablet_service.cc:1514] Processing DeleteTablet for tablet caa1a8173b6643aaa5dc19d324830dae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:32 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:51296
I20260806 02:12:32.219287 20796 tablet_service.cc:1514] Processing DeleteTablet for tablet caa1a8173b6643aaa5dc19d324830dae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:32 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:43976
I20260806 02:12:32.219389 20940 tablet_service.cc:1514] Processing DeleteTablet for tablet caa1a8173b6643aaa5dc19d324830dae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:32 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:42598
I20260806 02:12:32.219468 21123 tablet_replica.cc:330] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd: stopping tablet replica
I20260806 02:12:32.219467 21122 tablet_replica.cc:330] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98: stopping tablet replica
I20260806 02:12:32.219583 21124 tablet_replica.cc:330] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4: stopping tablet replica
I20260806 02:12:32.219619 21122 raft_consensus.cc:2229] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:12:32.219663 21123 raft_consensus.cc:2229] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:32.219703 21124 raft_consensus.cc:2229] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:32.219789 21123 raft_consensus.cc:2258] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:32.219821 21124 raft_consensus.cc:2258] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:32.219905 21122 raft_consensus.cc:2258] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:32.220583 21124 ts_tablet_manager.cc:1905] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:32.220583 21123 ts_tablet_manager.cc:1905] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:32.221086 21122 ts_tablet_manager.cc:1905] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:32.223316 21124 ts_tablet_manager.cc:1918] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:12:32.223390 21124 log.cc:1196] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/caa1a8173b6643aaa5dc19d324830dae
I20260806 02:12:32.223366 21123 ts_tablet_manager.cc:1918] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:12:32.223452 21123 log.cc:1196] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/caa1a8173b6643aaa5dc19d324830dae
I20260806 02:12:32.223695 21124 ts_tablet_manager.cc:1939] T caa1a8173b6643aaa5dc19d324830dae P f42bbabc6cd44ff5b16e46849184a3e4: Deleting consensus metadata
I20260806 02:12:32.223729 21122 ts_tablet_manager.cc:1918] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:12:32.223788 21123 ts_tablet_manager.cc:1939] T caa1a8173b6643aaa5dc19d324830dae P 9145b72ee2554e6eaf55fec53d59a7fd: Deleting consensus metadata
I20260806 02:12:32.223810 21122 log.cc:1196] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-0/wal/wals/caa1a8173b6643aaa5dc19d324830dae
I20260806 02:12:32.224215 21122 ts_tablet_manager.cc:1939] T caa1a8173b6643aaa5dc19d324830dae P 45bf89be760448f39b7368267db96f98: Deleting consensus metadata
I20260806 02:12:32.224215 20501 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:12:32.224507 20485 catalog_manager.cc:4841] TS 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66:41165): tablet caa1a8173b6643aaa5dc19d324830dae (table test-table [id=9fad1f5122334ba9be03e1a72ad71ff4]) successfully deleted
W20260806 02:12:32.224644 20501 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:32.225073 20488 catalog_manager.cc:4841] TS 45bf89be760448f39b7368267db96f98 (127.18.70.65:34163): tablet caa1a8173b6643aaa5dc19d324830dae (table test-table [id=9fad1f5122334ba9be03e1a72ad71ff4]) successfully deleted
I20260806 02:12:32.225687 20487 catalog_manager.cc:4841] TS f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567): tablet caa1a8173b6643aaa5dc19d324830dae (table test-table [id=9fad1f5122334ba9be03e1a72ad71ff4]) successfully deleted
I20260806 02:12:32.229168 20644 tablet_service.cc:1467] Processing CreateTablet for tablet d8bae3eb163b4230a49cc5e194d03b68 (DEFAULT_TABLE table=test-table [id=7bb29a9c406e47a995126b5e7b5ff9e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:32.229295 20796 tablet_service.cc:1467] Processing CreateTablet for tablet d8bae3eb163b4230a49cc5e194d03b68 (DEFAULT_TABLE table=test-table [id=7bb29a9c406e47a995126b5e7b5ff9e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:32.229439 20644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8bae3eb163b4230a49cc5e194d03b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:32.229441 20940 tablet_service.cc:1467] Processing CreateTablet for tablet d8bae3eb163b4230a49cc5e194d03b68 (DEFAULT_TABLE table=test-table [id=7bb29a9c406e47a995126b5e7b5ff9e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:32.229538 20796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8bae3eb163b4230a49cc5e194d03b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:32.229653 20940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8bae3eb163b4230a49cc5e194d03b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:32.231768 21134 tablet_bootstrap.cc:492] T d8bae3eb163b4230a49cc5e194d03b68 P 45bf89be760448f39b7368267db96f98: Bootstrap starting.
I20260806 02:12:32.231807 21135 tablet_bootstrap.cc:492] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4: Bootstrap starting.
I20260806 02:12:32.232577 21134 tablet_bootstrap.cc:654] T d8bae3eb163b4230a49cc5e194d03b68 P 45bf89be760448f39b7368267db96f98: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:32.232615 21135 tablet_bootstrap.cc:654] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:32.233222 21133 tablet_bootstrap.cc:492] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd: Bootstrap starting.
I20260806 02:12:32.233771 21134 tablet_bootstrap.cc:492] T d8bae3eb163b4230a49cc5e194d03b68 P 45bf89be760448f39b7368267db96f98: No bootstrap required, opened a new log
I20260806 02:12:32.233855 21134 ts_tablet_manager.cc:1397] T d8bae3eb163b4230a49cc5e194d03b68 P 45bf89be760448f39b7368267db96f98: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 02:12:32.233878 21133 tablet_bootstrap.cc:654] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:32.234243 21134 raft_consensus.cc:348] T d8bae3eb163b4230a49cc5e194d03b68 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.234354 21134 raft_consensus.cc:374] T d8bae3eb163b4230a49cc5e194d03b68 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:32.234390 21134 raft_consensus.cc:729] T d8bae3eb163b4230a49cc5e194d03b68 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45bf89be760448f39b7368267db96f98, State: Initialized, Role: FOLLOWER
I20260806 02:12:32.234522 21134 consensus_queue.cc:260] T d8bae3eb163b4230a49cc5e194d03b68 P 45bf89be760448f39b7368267db96f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.234795 21134 ts_tablet_manager.cc:1428] T d8bae3eb163b4230a49cc5e194d03b68 P 45bf89be760448f39b7368267db96f98: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:32.235138 21135 tablet_bootstrap.cc:492] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4: No bootstrap required, opened a new log
I20260806 02:12:32.235216 21135 ts_tablet_manager.cc:1397] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 02:12:32.235755 21133 tablet_bootstrap.cc:492] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd: No bootstrap required, opened a new log
I20260806 02:12:32.235852 21133 ts_tablet_manager.cc:1397] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 02:12:32.236214 21133 raft_consensus.cc:348] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.236311 21133 raft_consensus.cc:374] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:32.236332 21133 raft_consensus.cc:729] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9145b72ee2554e6eaf55fec53d59a7fd, State: Initialized, Role: FOLLOWER
I20260806 02:12:32.236446 21133 consensus_queue.cc:260] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.236734 21133 ts_tablet_manager.cc:1428] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:32.237475 21135 raft_consensus.cc:348] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.237622 21135 raft_consensus.cc:374] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:32.237655 21135 raft_consensus.cc:729] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f42bbabc6cd44ff5b16e46849184a3e4, State: Initialized, Role: FOLLOWER
I20260806 02:12:32.237779 21135 consensus_queue.cc:260] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.238044 21135 ts_tablet_manager.cc:1428] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 02:12:32.240016 21139 raft_consensus.cc:482] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:32.240128 21139 raft_consensus.cc:504] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.240502 21139 leader_election.cc:288] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45bf89be760448f39b7368267db96f98 (127.18.70.65:34163), f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567)
I20260806 02:12:32.240803 20665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8bae3eb163b4230a49cc5e194d03b68" candidate_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45bf89be760448f39b7368267db96f98" is_pre_election: true
I20260806 02:12:32.240922 20665 raft_consensus.cc:2454] T d8bae3eb163b4230a49cc5e194d03b68 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9145b72ee2554e6eaf55fec53d59a7fd in term 0.
I20260806 02:12:32.240948 20960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8bae3eb163b4230a49cc5e194d03b68" candidate_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" is_pre_election: true
I20260806 02:12:32.241065 20960 raft_consensus.cc:2454] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9145b72ee2554e6eaf55fec53d59a7fd in term 0.
I20260806 02:12:32.241323 20746 leader_election.cc:302] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45bf89be760448f39b7368267db96f98, 9145b72ee2554e6eaf55fec53d59a7fd; no voters:
I20260806 02:12:32.241475 21139 raft_consensus.cc:2781] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:32.241528 21139 raft_consensus.cc:482] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:32.241575 21139 raft_consensus.cc:3037] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:32.242458 21139 raft_consensus.cc:504] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.242825 21139 leader_election.cc:288] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [CANDIDATE]: Term 1 election: Requested vote from peers 45bf89be760448f39b7368267db96f98 (127.18.70.65:34163), f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567)
I20260806 02:12:32.243090 20665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8bae3eb163b4230a49cc5e194d03b68" candidate_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45bf89be760448f39b7368267db96f98"
I20260806 02:12:32.243151 20960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8bae3eb163b4230a49cc5e194d03b68" candidate_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f42bbabc6cd44ff5b16e46849184a3e4"
I20260806 02:12:32.243232 20665 raft_consensus.cc:3037] T d8bae3eb163b4230a49cc5e194d03b68 P 45bf89be760448f39b7368267db96f98 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:32.243275 20960 raft_consensus.cc:3037] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:32.244117 20960 raft_consensus.cc:2454] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9145b72ee2554e6eaf55fec53d59a7fd in term 1.
I20260806 02:12:32.244292 20665 raft_consensus.cc:2454] T d8bae3eb163b4230a49cc5e194d03b68 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9145b72ee2554e6eaf55fec53d59a7fd in term 1.
I20260806 02:12:32.244412 20745 leader_election.cc:302] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9145b72ee2554e6eaf55fec53d59a7fd, f42bbabc6cd44ff5b16e46849184a3e4; no voters:
I20260806 02:12:32.244625 21139 raft_consensus.cc:2781] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:32.244803 21139 raft_consensus.cc:686] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 LEADER]: Becoming Leader. State: Replica: 9145b72ee2554e6eaf55fec53d59a7fd, State: Running, Role: LEADER
I20260806 02:12:32.244937 21139 consensus_queue.cc:237] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } }
I20260806 02:12:32.246495 20502 catalog_manager.cc:5495] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66). New cstate: current_term: 1 leader_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9145b72ee2554e6eaf55fec53d59a7fd" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 41165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 } health_report { overall_health: UNKNOWN } } }
I20260806 02:12:32.262465 21154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:32.258381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43988 (local address 127.18.70.66:41165)
0806 02:12:32.258697 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:32.258705 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:32.258717 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:12:32.258767 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:32.258769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:32.258853 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:32.258934 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:32.259242 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:32.260060 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:32.261227 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:32.261581 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:32.261674 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:32.262006 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:32.262087 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:32.262153 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:32.262291 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 02:12:32.262341 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":129,"threads_started":1}
I20260806 02:12:32.269371 20665 raft_consensus.cc:1261] T d8bae3eb163b4230a49cc5e194d03b68 P 45bf89be760448f39b7368267db96f98 [term 1 FOLLOWER]: Refusing update from remote peer 9145b72ee2554e6eaf55fec53d59a7fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:32.269371 20960 raft_consensus.cc:1261] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Refusing update from remote peer 9145b72ee2554e6eaf55fec53d59a7fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:32.269747 21152 consensus_queue.cc:1035] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f42bbabc6cd44ff5b16e46849184a3e4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:32.269960 21139 consensus_queue.cc:1035] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "45bf89be760448f39b7368267db96f98" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 34163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 02:12:32.289251 21105 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 02:12:33.841549 20502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:33.841710 20502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:33.844075 20502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e27fa7ffa8a8472b9819c1dee9e1723d: Sending DeleteTablet for 3 replicas of tablet d8bae3eb163b4230a49cc5e194d03b68
I20260806 02:12:33.844590 20796 tablet_service.cc:1514] Processing DeleteTablet for tablet d8bae3eb163b4230a49cc5e194d03b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:33 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:43976
I20260806 02:12:33.844591 20643 tablet_service.cc:1514] Processing DeleteTablet for tablet d8bae3eb163b4230a49cc5e194d03b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:33 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:51296
I20260806 02:12:33.844702 20940 tablet_service.cc:1514] Processing DeleteTablet for tablet d8bae3eb163b4230a49cc5e194d03b68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:33 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:42598
I20260806 02:12:33.844930 21183 tablet_replica.cc:330] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd: stopping tablet replica
I20260806 02:12:33.845031 21185 tablet_replica.cc:330] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4: stopping tablet replica
I20260806 02:12:33.845057 21183 raft_consensus.cc:2229] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:12:33.845187 21185 raft_consensus.cc:2229] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:33.845218 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 20577
I20260806 02:12:33.845250 21183 raft_consensus.cc:2258] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:33.846933 21185 raft_consensus.cc:2258] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:33.847724 21183 ts_tablet_manager.cc:1905] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:33.848140 21185 ts_tablet_manager.cc:1905] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:33.850818 21185 ts_tablet_manager.cc:1918] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 02:12:33.850908 21185 log.cc:1196] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/d8bae3eb163b4230a49cc5e194d03b68
I20260806 02:12:33.851188 21185 ts_tablet_manager.cc:1939] T d8bae3eb163b4230a49cc5e194d03b68 P f42bbabc6cd44ff5b16e46849184a3e4: Deleting consensus metadata
I20260806 02:12:33.851210 21183 ts_tablet_manager.cc:1918] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 02:12:33.851310 21183 log.cc:1196] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/d8bae3eb163b4230a49cc5e194d03b68
I20260806 02:12:33.851639 21183 ts_tablet_manager.cc:1939] T d8bae3eb163b4230a49cc5e194d03b68 P 9145b72ee2554e6eaf55fec53d59a7fd: Deleting consensus metadata
I20260806 02:12:33.851934 20487 catalog_manager.cc:4841] TS f42bbabc6cd44ff5b16e46849184a3e4 (127.18.70.67:36567): tablet d8bae3eb163b4230a49cc5e194d03b68 (table test-table [id=7bb29a9c406e47a995126b5e7b5ff9e3]) successfully deleted
I20260806 02:12:33.852298 20485 catalog_manager.cc:4841] TS 9145b72ee2554e6eaf55fec53d59a7fd (127.18.70.66:41165): tablet d8bae3eb163b4230a49cc5e194d03b68 (table test-table [id=7bb29a9c406e47a995126b5e7b5ff9e3]) successfully deleted
W20260806 02:12:33.857339 20488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:34163 (error 108)
I20260806 02:12:33.857649 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 20726
I20260806 02:12:33.859874 21187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:33.859197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:34163 (local address 127.0.0.1:51328)
0806 02:12:33.859612 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:33.859728 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:34163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":168,"threads_started":1}
W20260806 02:12:33.859987 20488 catalog_manager.cc:4584] TS 45bf89be760448f39b7368267db96f98 (127.18.70.65:34163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8bae3eb163b4230a49cc5e194d03b68: Network error: Client connection negotiation failed: client connection to 127.18.70.65:34163: connect: Connection refused (error 111)
I20260806 02:12:33.866549 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 20874
I20260806 02:12:33.874194 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 20468
2026-08-06T02:12:33Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6504 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:12:33 dist-test-slave-6k22 krb5kdc[21190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:12:33 dist-test-slave-6k22 krb5kdc[21190](info): set up 2 sockets
Aug 06 02:12:33 dist-test-slave-6k22 krb5kdc[21190](info): commencing operation
krb5kdc: starting...
W20260806 02:12:35.061151 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:12:35 dist-test-slave-6k22 krb5kdc[21190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:12:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:12:35.183350 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:45463
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:34681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:45463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:35.293565 21204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:35.293836 21204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:35.293910 21204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:35.297835 21204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:35.297914 21204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:35.297952 21204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:35.298008 21204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:35.298045 21204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:35.303350 21204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:34681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:45463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:45463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:35.304363 21204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:35.305339 21204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 02:12:35.314126 21204 server_base.cc:1034] running on GCE node
W20260806 02:12:35.314030 21209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:35.314071 21212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:35.314086 21210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:35.314852 21204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:35.315986 21204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:35.317176 21204 hybrid_clock.cc:648] HybridClock initialized: now 1785982355317145 us; error 42 us; skew 500 ppm
Aug 06 02:12:35 dist-test-slave-6k22 krb5kdc[21190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982355, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:35.320238 21204 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:12:35.321091 21204 webserver.cc:460] Webserver started at http://127.18.70.126:37715/ using document root <none> and password file <none>
I20260806 02:12:35.321661 21204 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:35.321755 21204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:35.321996 21204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:35.324090 21204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "6237dd045fb042e1b8800ce61edf3991"
format_stamp: "Formatted at 2026-08-06 02:12:35 on dist-test-slave-6k22"
server_key: "c090ac53165297014965d4cb4a02a279"
server_key_iv: "5e9d0ea82e73658746ffe7e11278da4b"
server_key_version: "encryptionkey@0"
I20260806 02:12:35.324677 21204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "6237dd045fb042e1b8800ce61edf3991"
format_stamp: "Formatted at 2026-08-06 02:12:35 on dist-test-slave-6k22"
server_key: "c090ac53165297014965d4cb4a02a279"
server_key_iv: "5e9d0ea82e73658746ffe7e11278da4b"
server_key_version: "encryptionkey@0"
I20260806 02:12:35.328506 21204 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 02:12:35.330883 21219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:35.332108 21204 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:35.332264 21204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "6237dd045fb042e1b8800ce61edf3991"
format_stamp: "Formatted at 2026-08-06 02:12:35 on dist-test-slave-6k22"
server_key: "c090ac53165297014965d4cb4a02a279"
server_key_iv: "5e9d0ea82e73658746ffe7e11278da4b"
server_key_version: "encryptionkey@0"
I20260806 02:12:35.332374 21204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:35.358590 21204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:35.361932 21204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:35.362190 21204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:35.369484 21204 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:45463
I20260806 02:12:35.369508 21272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:45463 every 8 connection(s)
I20260806 02:12:35.370452 21204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:35.373175 21273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:35.378659 21273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991: Bootstrap starting.
I20260806 02:12:35.379503 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 21204
I20260806 02:12:35.379667 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:35.379940 18713 external_mini_cluster.cc:1428] Setting key eaba86793c78bd2b634ffee160288853
I20260806 02:12:35.381443 21273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:35.382159 21273 log.cc:824] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:35.384368 21273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991: No bootstrap required, opened a new log
Aug 06 02:12:35 dist-test-slave-6k22 krb5kdc[21190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:35.388929 21273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6237dd045fb042e1b8800ce61edf3991" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45463 } }
I20260806 02:12:35.389171 21273 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:35.389240 21273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6237dd045fb042e1b8800ce61edf3991, State: Initialized, Role: FOLLOWER
I20260806 02:12:35.389737 21273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6237dd045fb042e1b8800ce61edf3991" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45463 } }
I20260806 02:12:35.389916 21273 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:35.389994 21273 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:35.390110 21273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:35.391294 21273 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6237dd045fb042e1b8800ce61edf3991" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45463 } }
I20260806 02:12:35.391693 21273 leader_election.cc:302] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6237dd045fb042e1b8800ce61edf3991; no voters:
I20260806 02:12:35.392059 21273 leader_election.cc:288] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:35.392124 21278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:35.392349 21278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [term 1 LEADER]: Becoming Leader. State: Replica: 6237dd045fb042e1b8800ce61edf3991, State: Running, Role: LEADER
I20260806 02:12:35.392710 21278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6237dd045fb042e1b8800ce61edf3991" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45463 } }
I20260806 02:12:35.393419 21273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:35.394752 21280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6237dd045fb042e1b8800ce61edf3991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6237dd045fb042e1b8800ce61edf3991" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45463 } } }
I20260806 02:12:35.395141 21280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:35.395341 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:35.381076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49186 (local address 127.18.70.126:45463)
0806 02:12:35.381506 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:35.381522 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:35.381586 (+ 64us) server_negotiation.cc:407] Connection header received
0806 02:12:35.382225 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:35.382242 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:35.382673 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:35.383014 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:35.383948 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:35.385037 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:35.386189 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:35.386644 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:35.386754 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:35.389686 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:35.389692 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:35.389708 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:35.389746 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:35.391742 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:35.392280 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:35.392285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:35.392290 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:35.392452 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:35.392797 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:35.392800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:35.392802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:35.393250 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:35.393393 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:35.393910 (+ 517us) server_negotiation.cc:299] Negotiation successful
0806 02:12:35.394233 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":132,"threads_started":1}
I20260806 02:12:35.395046 21279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6237dd045fb042e1b8800ce61edf3991. Latest consensus state: current_term: 1 leader_uuid: "6237dd045fb042e1b8800ce61edf3991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6237dd045fb042e1b8800ce61edf3991" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45463 } } }
I20260806 02:12:35.395449 21279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991 [sys.catalog]: This master's current role is: LEADER
W20260806 02:12:35.398681 21293 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:12:35.398747 21293 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:12:35.398826 21292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:35.399544 21292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:35.405436 21292 catalog_manager.cc:1347] Generated new cluster ID: 0e95ac09ea8f46bf8861bd438eb2330b
I20260806 02:12:35.405510 21292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:35.411599 21292 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:35.412781 21292 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:35.416872 21292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6237dd045fb042e1b8800ce61edf3991: Generated new TSK 0
I20260806 02:12:35.417548 21292 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:12:35.454466 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45463
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:34681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:35.578112 21299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:35.578401 21299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:35.578449 21299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:35.582582 21299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:35.582645 21299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:35.582705 21299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:12:35.588511 21299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:34681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:35.589526 21299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:35.590622 21299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:35.600394 21304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:35.600383 21307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:35.600379 21305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:35.600782 21299 server_base.cc:1034] running on GCE node
I20260806 02:12:35.601192 21299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:35.601841 21299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:35.603027 21299 hybrid_clock.cc:648] HybridClock initialized: now 1785982355602996 us; error 44 us; skew 500 ppm
Aug 06 02:12:35 dist-test-slave-6k22 krb5kdc[21190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982355, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:35.605726 21299 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:12:35.606550 21299 webserver.cc:460] Webserver started at http://127.18.70.65:44043/ using document root <none> and password file <none>
I20260806 02:12:35.607146 21299 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:35.607244 21299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:35.607492 21299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:35.609580 21299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "9bdbe5e847b241bd819ecc528f785476"
format_stamp: "Formatted at 2026-08-06 02:12:35 on dist-test-slave-6k22"
server_key: "efa93b629a18da5904dbae418ef1191d"
server_key_iv: "caee4a1219eeb9d787cb5f210cb3425b"
server_key_version: "encryptionkey@0"
I20260806 02:12:35.610134 21299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "9bdbe5e847b241bd819ecc528f785476"
format_stamp: "Formatted at 2026-08-06 02:12:35 on dist-test-slave-6k22"
server_key: "efa93b629a18da5904dbae418ef1191d"
server_key_iv: "caee4a1219eeb9d787cb5f210cb3425b"
server_key_version: "encryptionkey@0"
I20260806 02:12:35.613802 21299 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:12:35.616240 21315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:35.617476 21299 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 02:12:35.617609 21299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "9bdbe5e847b241bd819ecc528f785476"
format_stamp: "Formatted at 2026-08-06 02:12:35 on dist-test-slave-6k22"
server_key: "efa93b629a18da5904dbae418ef1191d"
server_key_iv: "caee4a1219eeb9d787cb5f210cb3425b"
server_key_version: "encryptionkey@0"
I20260806 02:12:35.617750 21299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:35.635401 21299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:35.638480 21299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:35.638687 21299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:35.639287 21299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:35.640148 21299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:35.640219 21299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:35.640287 21299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:35.640331 21299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:35.649184 21299 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:35165
I20260806 02:12:35.649212 21428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:35165 every 8 connection(s)
I20260806 02:12:35.650102 21299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:12:35.651540 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 21299
I20260806 02:12:35.651691 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:12:35.652036 18713 external_mini_cluster.cc:1428] Setting key c5831148b032f0732ef1846ba4db3337
Aug 06 02:12:35 dist-test-slave-6k22 krb5kdc[21190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982355, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
I20260806 02:12:35.665977 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:35.652018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:35229 (local address 127.18.70.126:45463)
0806 02:12:35.652174 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:35.652184 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:35.653167 (+ 983us) server_negotiation.cc:407] Connection header received
0806 02:12:35.654601 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:35.654607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:35.654668 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:35.654775 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:35.656255 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:35.657148 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:35.658996 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:35.659318 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:35.659406 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:35.662339 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:35.662343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:35.662351 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:35.662375 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:35.663924 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:35.664434 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:35.664437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:35.664439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:35.664523 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:35.664866 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:35.664871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:35.664872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:35.665052 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:35.665156 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:35.665662 (+ 506us) server_negotiation.cc:299] Negotiation successful
0806 02:12:35.665797 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
I20260806 02:12:35.666935 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:35.652448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45463 (local address 127.18.70.65:35229)
0806 02:12:35.652997 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:35.653056 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:35.654381 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:35.654943 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:35.654949 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:35.655299 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:35.656038 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:35.656060 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:35.657300 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:35.657307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:35.658849 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:35.658865 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:35.659494 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:35.659501 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:35.659771 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:35.660401 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:35.660428 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:35.662163 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:35.664038 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:35.664043 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:35.664059 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:35.664325 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:35.664618 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:35.664622 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:35.664624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:35.664770 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:35.665172 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:35.665180 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:35.665427 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 02:12:35.665621 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 02:12:35.665943 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":160,"threads_started":1}
I20260806 02:12:35.668351 21429 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45463
I20260806 02:12:35.668624 21429 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:35.669234 21429 heartbeater.cc:511] Master 127.18.70.126:45463 requested a full tablet report, sending...
I20260806 02:12:35.670894 21236 ts_manager.cc:194] Registered new tserver with Master: 9bdbe5e847b241bd819ecc528f785476 (127.18.70.65:35165)
I20260806 02:12:35.672116 21236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:35229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:12:35.683001 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45463
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:34681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:35.795893 21434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:35.796155 21434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:35.796240 21434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:35.800315 21434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:35.800386 21434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:35.800472 21434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:12:35.805474 21434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:34681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:35.806450 21434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:35.807461 21434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:35.816880 21439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:35.816885 21440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:35.817091 21434 server_base.cc:1034] running on GCE node
W20260806 02:12:35.816902 21442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:35.817607 21434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:35.818257 21434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:35.819447 21434 hybrid_clock.cc:648] HybridClock initialized: now 1785982355819412 us; error 46 us; skew 500 ppm
Aug 06 02:12:35 dist-test-slave-6k22 krb5kdc[21190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982355, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:35.822310 21434 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:12:35.823072 21434 webserver.cc:460] Webserver started at http://127.18.70.66:42701/ using document root <none> and password file <none>
I20260806 02:12:35.823661 21434 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:35.823726 21434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:35.823902 21434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:35.825821 21434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "f78049158b6e4d3997f0fd4a1435803f"
format_stamp: "Formatted at 2026-08-06 02:12:35 on dist-test-slave-6k22"
server_key: "3d84fa0df6620599f3d83de656170e49"
server_key_iv: "1fe42bcd3bb26f8299e02952f34945ee"
server_key_version: "encryptionkey@0"
I20260806 02:12:35.826320 21434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "f78049158b6e4d3997f0fd4a1435803f"
format_stamp: "Formatted at 2026-08-06 02:12:35 on dist-test-slave-6k22"
server_key: "3d84fa0df6620599f3d83de656170e49"
server_key_iv: "1fe42bcd3bb26f8299e02952f34945ee"
server_key_version: "encryptionkey@0"
I20260806 02:12:35.829941 21434 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:12:35.832396 21450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:35.833536 21434 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:35.833645 21434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "f78049158b6e4d3997f0fd4a1435803f"
format_stamp: "Formatted at 2026-08-06 02:12:35 on dist-test-slave-6k22"
server_key: "3d84fa0df6620599f3d83de656170e49"
server_key_iv: "1fe42bcd3bb26f8299e02952f34945ee"
server_key_version: "encryptionkey@0"
I20260806 02:12:35.833724 21434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:35.854413 21434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:35.857286 21434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:35.857426 21434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:35.857954 21434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:35.858728 21434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:35.858783 21434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:35.858820 21434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:35.858835 21434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:35.867919 21434 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:33005
I20260806 02:12:35.867949 21564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:33005 every 8 connection(s)
I20260806 02:12:35.868799 21434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:12:35.869948 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 21434
I20260806 02:12:35.870055 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:12:35.870412 18713 external_mini_cluster.cc:1428] Setting key 17aed027dc482fb3d9f217cc7c3d2463
Aug 06 02:12:35 dist-test-slave-6k22 krb5kdc[21190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982355, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
I20260806 02:12:35.884476 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:35.870668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:45595 (local address 127.18.70.126:45463)
0806 02:12:35.870846 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:35.870850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:35.871821 (+ 971us) server_negotiation.cc:407] Connection header received
0806 02:12:35.873723 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:35.873725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:35.873770 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:35.873832 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:35.875090 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:35.875868 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:35.877701 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:35.877913 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:35.877984 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:35.880817 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:35.880824 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:35.880827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:35.880856 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:35.882366 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:35.882813 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:35.882815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:35.882817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:35.882939 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:35.883290 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:35.883295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:35.883297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:35.883473 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:35.883631 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:35.884165 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 02:12:35.884282 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
I20260806 02:12:35.885389 21567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:35.871117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45463 (local address 127.18.70.66:45595)
0806 02:12:35.871654 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:35.871710 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:35.873477 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:35.874041 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:35.874053 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:35.874452 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:35.874932 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:35.874945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:35.876011 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:35.876017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:35.877565 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:35.877578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:35.878082 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:35.878089 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:35.878373 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:35.878999 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:35.879026 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:35.880643 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:35.882468 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:35.882473 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:35.882484 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:35.882715 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:35.883043 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:35.883046 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:35.883048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:35.883184 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:35.883632 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:35.883640 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:35.883873 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 02:12:35.884095 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 02:12:35.884415 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":117,"threads_started":1}
I20260806 02:12:35.886617 21565 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45463
I20260806 02:12:35.886883 21565 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:35.887451 21565 heartbeater.cc:511] Master 127.18.70.126:45463 requested a full tablet report, sending...
I20260806 02:12:35.888654 21236 ts_manager.cc:194] Registered new tserver with Master: f78049158b6e4d3997f0fd4a1435803f (127.18.70.66:33005)
I20260806 02:12:35.889458 21236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:45595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:12:35.902292 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45463
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:34681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:36.016770 21570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:36.017024 21570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:36.017091 21570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:36.022106 21570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:36.022246 21570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:36.022320 21570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:12:36.028234 21570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:34681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:36.029181 21570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:36.030109 21570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:36.040082 21578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:36.040093 21576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:36.040148 21575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:36.040309 21570 server_base.cc:1034] running on GCE node
I20260806 02:12:36.040913 21570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:36.041560 21570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:36.042788 21570 hybrid_clock.cc:648] HybridClock initialized: now 1785982356042770 us; error 24 us; skew 500 ppm
Aug 06 02:12:36 dist-test-slave-6k22 krb5kdc[21190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982356, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:36.045910 21570 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:12:36.046890 21570 webserver.cc:460] Webserver started at http://127.18.70.67:46367/ using document root <none> and password file <none>
I20260806 02:12:36.047457 21570 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:36.047554 21570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:36.047807 21570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:36.049799 21570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "f2e425c8e49b40c8ac771df52d22dc7c"
format_stamp: "Formatted at 2026-08-06 02:12:36 on dist-test-slave-6k22"
server_key: "1681b66afd88bdde7636a43e603b6a0e"
server_key_iv: "a1617c9a0b2d791e8d13aec3faaff704"
server_key_version: "encryptionkey@0"
I20260806 02:12:36.050340 21570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "f2e425c8e49b40c8ac771df52d22dc7c"
format_stamp: "Formatted at 2026-08-06 02:12:36 on dist-test-slave-6k22"
server_key: "1681b66afd88bdde7636a43e603b6a0e"
server_key_iv: "a1617c9a0b2d791e8d13aec3faaff704"
server_key_version: "encryptionkey@0"
I20260806 02:12:36.054131 21570 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 02:12:36.056531 21585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:36.057790 21570 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:36.057943 21570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "f2e425c8e49b40c8ac771df52d22dc7c"
format_stamp: "Formatted at 2026-08-06 02:12:36 on dist-test-slave-6k22"
server_key: "1681b66afd88bdde7636a43e603b6a0e"
server_key_iv: "a1617c9a0b2d791e8d13aec3faaff704"
server_key_version: "encryptionkey@0"
I20260806 02:12:36.058073 21570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:36.071022 21570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:36.074534 21570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:36.074715 21570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:36.075275 21570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:36.076177 21570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:36.076229 21570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:36.076272 21570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:36.076297 21570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:36.085698 21570 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:44073
I20260806 02:12:36.085758 21699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:44073 every 8 connection(s)
I20260806 02:12:36.086673 21570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:12:36.088832 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 21570
I20260806 02:12:36.089007 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:12:36.089325 18713 external_mini_cluster.cc:1428] Setting key 3cab9c40d7a297f45c1c8e144a114024
Aug 06 02:12:36 dist-test-slave-6k22 krb5kdc[21190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982356, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:36.100543 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:36.088678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:51877 (local address 127.18.70.126:45463)
0806 02:12:36.088860 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:36.088865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:36.089875 (+ 1010us) server_negotiation.cc:407] Connection header received
0806 02:12:36.091269 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:36.091273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:36.091333 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:36.091440 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:36.092665 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:36.093245 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:36.094539 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:36.094803 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:36.094925 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:36.097420 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:36.097423 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:36.097425 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:36.097449 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:36.098714 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:36.099182 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:36.099184 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:36.099186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:36.099259 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:36.099600 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:36.099604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:36.099605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:36.099797 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:36.099876 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:36.100330 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 02:12:36.100423 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 02:12:36.101344 21702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:36.089214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45463 (local address 127.18.70.67:51877)
0806 02:12:36.089726 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:36.089782 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:36.091038 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:36.091657 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:36.091665 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:36.092017 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:36.092502 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:36.092516 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:36.093398 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:36.093409 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:36.094406 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:36.094417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:36.094963 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:36.094967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:36.095168 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:36.095751 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:36.095770 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:36.097237 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:36.098830 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:36.098835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:36.098846 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:36.099096 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:36.099339 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:36.099341 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:36.099342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:36.099478 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:36.099889 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:36.099895 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:36.100118 (+ 223us) client_negotiation.cc:769] Sending connection context
0806 02:12:36.100282 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 02:12:36.100520 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":183,"threads_started":1}
I20260806 02:12:36.102764 21700 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45463
I20260806 02:12:36.103045 21700 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:36.103680 21700 heartbeater.cc:511] Master 127.18.70.126:45463 requested a full tablet report, sending...
I20260806 02:12:36.104686 21236 ts_manager.cc:194] Registered new tserver with Master: f2e425c8e49b40c8ac771df52d22dc7c (127.18.70.67:44073)
I20260806 02:12:36.105243 21236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:51877
I20260806 02:12:36.115147 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 02:12:36.133641 21276 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:12:36.132268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.18.70.126:45463)
0806 02:12:36.132473 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:36.132478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:36.132540 (+ 62us) server_negotiation.cc:407] Connection header received
0806 02:12:36.132672 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:36.132674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:36.132717 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:36.132786 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:36.133496 (+ 710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49196: BlockingRecv error: recv got EOF from 127.0.0.1:49196 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 02:12:36.134667 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 21299
I20260806 02:12:36.143021 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 21434
I20260806 02:12:36.149727 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 21570
I20260806 02:12:36.155877 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 21204
2026-08-06T02:12:36Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2282 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:12:36 dist-test-slave-6k22 krb5kdc[21712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:12:36 dist-test-slave-6k22 krb5kdc[21712](info): set up 2 sockets
Aug 06 02:12:36 dist-test-slave-6k22 krb5kdc[21712](info): commencing operation
krb5kdc: starting...
W20260806 02:12:37.225265 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:12:37 dist-test-slave-6k22 krb5kdc[21712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:12:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:12:37.347360 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:38245
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:42133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:38245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:37.456621 21725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:37.456853 21725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:37.456890 21725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:37.460721 21725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:37.460772 21725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:37.460791 21725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:37.460807 21725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:37.460822 21725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:37.465757 21725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:42133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:38245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:38245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:37.466727 21725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:37.467669 21725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:37.476282 21733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:37.476370 21725 server_base.cc:1034] running on GCE node
W20260806 02:12:37.476284 21730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:37.476286 21731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:37.477069 21725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:37.478138 21725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:37.479300 21725 hybrid_clock.cc:648] HybridClock initialized: now 1785982357479264 us; error 49 us; skew 500 ppm
Aug 06 02:12:37 dist-test-slave-6k22 krb5kdc[21712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982357, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:37.486232 21725 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:12:37.487147 21725 webserver.cc:460] Webserver started at http://127.18.70.126:42445/ using document root <none> and password file <none>
I20260806 02:12:37.487771 21725 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:37.487835 21725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:37.488072 21725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:37.490095 21725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "4ec75b4769a04f27b53a6788f40a60ba"
format_stamp: "Formatted at 2026-08-06 02:12:37 on dist-test-slave-6k22"
server_key: "0749c1c5d739b00bb158ac7f929058fb"
server_key_iv: "9c211f1f8c165172d7f84dfc4639bbf8"
server_key_version: "encryptionkey@0"
I20260806 02:12:37.490639 21725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "4ec75b4769a04f27b53a6788f40a60ba"
format_stamp: "Formatted at 2026-08-06 02:12:37 on dist-test-slave-6k22"
server_key: "0749c1c5d739b00bb158ac7f929058fb"
server_key_iv: "9c211f1f8c165172d7f84dfc4639bbf8"
server_key_version: "encryptionkey@0"
I20260806 02:12:37.494426 21725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 02:12:37.496769 21740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:37.497968 21725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:37.498118 21725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "4ec75b4769a04f27b53a6788f40a60ba"
format_stamp: "Formatted at 2026-08-06 02:12:37 on dist-test-slave-6k22"
server_key: "0749c1c5d739b00bb158ac7f929058fb"
server_key_iv: "9c211f1f8c165172d7f84dfc4639bbf8"
server_key_version: "encryptionkey@0"
I20260806 02:12:37.498226 21725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:37.527916 21725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:37.530791 21725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:37.530994 21725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:37.538326 21725 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:38245
I20260806 02:12:37.538621 21796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:38245 every 8 connection(s)
I20260806 02:12:37.539355 21725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:37.542178 21797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:37.543795 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 21725
I20260806 02:12:37.543926 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:37.544206 18713 external_mini_cluster.cc:1428] Setting key 2d63ebeffd139a219b728655b8ba72d1
I20260806 02:12:37.548696 21797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba: Bootstrap starting.
I20260806 02:12:37.551313 21797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:37.552316 21797 log.cc:824] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba: Log is configured to *not* fsync() on all Append() calls
Aug 06 02:12:37 dist-test-slave-6k22 krb5kdc[21712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:37.554360 21797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba: No bootstrap required, opened a new log
I20260806 02:12:37.559172 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:37.545603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44444 (local address 127.18.70.126:38245)
0806 02:12:37.546033 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:37.546051 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:37.546087 (+ 36us) server_negotiation.cc:407] Connection header received
0806 02:12:37.546706 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:37.546722 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:37.547044 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:37.547329 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:37.548337 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:37.549355 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:37.550540 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:37.550984 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:37.551101 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:37.554091 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:37.554096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:37.554110 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:37.554136 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:37.556083 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:37.556500 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:37.556505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:37.556513 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:37.556660 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:37.556942 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:37.556945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:37.556947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:37.557495 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:37.557680 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:37.557899 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 02:12:37.558197 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":153,"threads_started":1}
I20260806 02:12:37.559211 21797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec75b4769a04f27b53a6788f40a60ba" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38245 } }
I20260806 02:12:37.559482 21797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:37.559548 21797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec75b4769a04f27b53a6788f40a60ba, State: Initialized, Role: FOLLOWER
I20260806 02:12:37.559986 21797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec75b4769a04f27b53a6788f40a60ba" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38245 } }
I20260806 02:12:37.560103 21797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:37.560190 21797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:37.560298 21797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:37.561408 21797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec75b4769a04f27b53a6788f40a60ba" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38245 } }
I20260806 02:12:37.561789 21797 leader_election.cc:302] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ec75b4769a04f27b53a6788f40a60ba; no voters:
I20260806 02:12:37.562178 21797 leader_election.cc:288] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:37.562270 21802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:37.562434 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:12:37.562492 21802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [term 1 LEADER]: Becoming Leader. State: Replica: 4ec75b4769a04f27b53a6788f40a60ba, State: Running, Role: LEADER
I20260806 02:12:37.562772 21802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec75b4769a04f27b53a6788f40a60ba" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38245 } }
I20260806 02:12:37.563360 21797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:37.563977 21802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ec75b4769a04f27b53a6788f40a60ba. Latest consensus state: current_term: 1 leader_uuid: "4ec75b4769a04f27b53a6788f40a60ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec75b4769a04f27b53a6788f40a60ba" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38245 } } }
I20260806 02:12:37.564095 21802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:37.564397 21804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ec75b4769a04f27b53a6788f40a60ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec75b4769a04f27b53a6788f40a60ba" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38245 } } }
I20260806 02:12:37.564491 21804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba [sys.catalog]: This master's current role is: LEADER
W20260806 02:12:37.567996 21819 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:12:37.568090 21819 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:12:37.568209 21809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:37.569015 21809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:37.574457 21809 catalog_manager.cc:1347] Generated new cluster ID: fc9766effb464e4291f85606a53f9fb2
I20260806 02:12:37.574517 21809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:37.584065 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:37.578302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44460 (local address 127.18.70.126:38245)
0806 02:12:37.578413 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:37.578417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:37.578738 (+ 321us) server_negotiation.cc:407] Connection header received
0806 02:12:37.578884 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:37.578887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:37.578932 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:37.579017 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:37.579662 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:37.580480 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:37.581677 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:37.582018 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:37.582111 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:37.583052 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:37.583056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:37.583059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:37.583602 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:37.583634 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:37.583657 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:37.583744 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:37.583875 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 02:12:37.583931 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 02:12:37.597302 21809 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:37.598567 21809 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:37.608419 21809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ec75b4769a04f27b53a6788f40a60ba: Generated new TSK 0
I20260806 02:12:37.609186 21809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:12:37.619671 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 21725
2026-08-06T02:12:37Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1461 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:12:37 dist-test-slave-6k22 krb5kdc[21832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:12:37 dist-test-slave-6k22 krb5kdc[21832](info): set up 2 sockets
Aug 06 02:12:37 dist-test-slave-6k22 krb5kdc[21832](info): commencing operation
krb5kdc: starting...
W20260806 02:12:38.672678 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:12:38 dist-test-slave-6k22 krb5kdc[21832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:12:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:12:38.798115 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:45835
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:45835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:38.910897 21846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:38.911159 21846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:38.911229 21846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:38.915226 21846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:38.915295 21846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:38.915330 21846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:38.915364 21846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:38.915396 21846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:38.920598 21846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:45835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:45835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:38.921573 21846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:38.922581 21846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:38.930912 21851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:38.930912 21853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:38.930945 21855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:38.931110 21846 server_base.cc:1034] running on GCE node
I20260806 02:12:38.931759 21846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:38.932775 21846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:38.933938 21846 hybrid_clock.cc:648] HybridClock initialized: now 1785982358933912 us; error 39 us; skew 500 ppm
Aug 06 02:12:38 dist-test-slave-6k22 krb5kdc[21832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982358, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:38.936681 21846 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:12:38.937539 21846 webserver.cc:460] Webserver started at http://127.18.70.126:33705/ using document root <none> and password file <none>
I20260806 02:12:38.938063 21846 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:38.938120 21846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:38.938374 21846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:38.940428 21846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "04b5b82d3d4747c8b77abdd851cdef74"
format_stamp: "Formatted at 2026-08-06 02:12:38 on dist-test-slave-6k22"
server_key: "04ee01b50431e5a0b033eb4468f60acc"
server_key_iv: "8ae327c789f2a32fef1b080437533be6"
server_key_version: "encryptionkey@0"
I20260806 02:12:38.940968 21846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "04b5b82d3d4747c8b77abdd851cdef74"
format_stamp: "Formatted at 2026-08-06 02:12:38 on dist-test-slave-6k22"
server_key: "04ee01b50431e5a0b033eb4468f60acc"
server_key_iv: "8ae327c789f2a32fef1b080437533be6"
server_key_version: "encryptionkey@0"
I20260806 02:12:38.944486 21846 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 02:12:38.946725 21863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:38.947896 21846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 02:12:38.948014 21846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "04b5b82d3d4747c8b77abdd851cdef74"
format_stamp: "Formatted at 2026-08-06 02:12:38 on dist-test-slave-6k22"
server_key: "04ee01b50431e5a0b033eb4468f60acc"
server_key_iv: "8ae327c789f2a32fef1b080437533be6"
server_key_version: "encryptionkey@0"
I20260806 02:12:38.948104 21846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:38.956420 21846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:38.959652 21846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:38.959838 21846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:38.967157 21846 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:45835
I20260806 02:12:38.967455 21916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:45835 every 8 connection(s)
I20260806 02:12:38.968117 21846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:38.970784 21917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:38.974313 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 21846
I20260806 02:12:38.974437 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:38.974768 18713 external_mini_cluster.cc:1428] Setting key 2ec42b9f2e1bcf8a9a19c16e42dc20e6
I20260806 02:12:38.976712 21917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74: Bootstrap starting.
I20260806 02:12:38.979244 21917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:38.980094 21917 log.cc:824] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:38.982151 21917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74: No bootstrap required, opened a new log
Aug 06 02:12:38 dist-test-slave-6k22 krb5kdc[21832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:38.986056 21917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b5b82d3d4747c8b77abdd851cdef74" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45835 } }
I20260806 02:12:38.986382 21917 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:38.986445 21917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04b5b82d3d4747c8b77abdd851cdef74, State: Initialized, Role: FOLLOWER
I20260806 02:12:38.986949 21917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b5b82d3d4747c8b77abdd851cdef74" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45835 } }
I20260806 02:12:38.987093 21917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:38.987190 21917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:38.987331 21917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:38.988209 21917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b5b82d3d4747c8b77abdd851cdef74" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45835 } }
I20260806 02:12:38.988569 21917 leader_election.cc:302] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04b5b82d3d4747c8b77abdd851cdef74; no voters:
I20260806 02:12:38.988888 21917 leader_election.cc:288] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:38.988953 21922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:38.989144 21922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [term 1 LEADER]: Becoming Leader. State: Replica: 04b5b82d3d4747c8b77abdd851cdef74, State: Running, Role: LEADER
I20260806 02:12:38.989296 21920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:38.976100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45134 (local address 127.18.70.126:45835)
0806 02:12:38.976562 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:38.976578 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:38.976613 (+ 35us) server_negotiation.cc:407] Connection header received
0806 02:12:38.977273 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:38.977290 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:38.977631 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:38.977906 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:38.978797 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:38.979889 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:38.980936 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:38.981370 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:38.981514 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:38.984240 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:38.984247 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:38.984261 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:38.984286 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:38.986026 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:38.986519 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:38.986527 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:38.986534 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:38.986658 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:38.986995 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:38.986999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:38.987000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:38.987592 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:38.987779 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:38.988089 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 02:12:38.988428 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":132,"threads_started":1}
I20260806 02:12:38.989480 21922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b5b82d3d4747c8b77abdd851cdef74" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45835 } }
I20260806 02:12:38.990118 21917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:38.990625 21922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04b5b82d3d4747c8b77abdd851cdef74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b5b82d3d4747c8b77abdd851cdef74" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45835 } } }
I20260806 02:12:38.990805 21922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:38.991070 21923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04b5b82d3d4747c8b77abdd851cdef74. Latest consensus state: current_term: 1 leader_uuid: "04b5b82d3d4747c8b77abdd851cdef74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b5b82d3d4747c8b77abdd851cdef74" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45835 } } }
I20260806 02:12:38.991226 21923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:38.991624 21928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:38.994344 21928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:39.000161 21928 catalog_manager.cc:1347] Generated new cluster ID: 050ff4822e124833af2acf64bfcc7f69
I20260806 02:12:39.000231 21928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:39.007548 21928 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:39.008461 21928 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:39.016928 21928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04b5b82d3d4747c8b77abdd851cdef74: Generated new TSK 0
I20260806 02:12:39.017524 21928 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:12:39.054397 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45835
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:39.165433 21943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:39.165659 21943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:39.165714 21943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:39.169276 21943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:39.169318 21943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:39.169414 21943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:12:39.174253 21943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:39.175136 21943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:39.176069 21943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:39.185266 21949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:39.185341 21950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:39.185474 21943 server_base.cc:1034] running on GCE node
W20260806 02:12:39.185474 21952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:39.185982 21943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:39.186565 21943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:39.187776 21943 hybrid_clock.cc:648] HybridClock initialized: now 1785982359187748 us; error 37 us; skew 500 ppm
Aug 06 02:12:39 dist-test-slave-6k22 krb5kdc[21832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:39.190608 21943 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:12:39.191635 21943 webserver.cc:460] Webserver started at http://127.18.70.65:42207/ using document root <none> and password file <none>
I20260806 02:12:39.192155 21943 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:39.192238 21943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:39.192449 21943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:39.194308 21943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "403b6a55febf432eb3101faacdab43f6"
format_stamp: "Formatted at 2026-08-06 02:12:39 on dist-test-slave-6k22"
server_key: "e8b085873db7d7fed07278f9f90cc3ea"
server_key_iv: "8b62012774e492c037bcd8672fa8bd7b"
server_key_version: "encryptionkey@0"
I20260806 02:12:39.194860 21943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "403b6a55febf432eb3101faacdab43f6"
format_stamp: "Formatted at 2026-08-06 02:12:39 on dist-test-slave-6k22"
server_key: "e8b085873db7d7fed07278f9f90cc3ea"
server_key_iv: "8b62012774e492c037bcd8672fa8bd7b"
server_key_version: "encryptionkey@0"
I20260806 02:12:39.198514 21943 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 02:12:39.200923 21959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:39.202054 21943 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:39.202201 21943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "403b6a55febf432eb3101faacdab43f6"
format_stamp: "Formatted at 2026-08-06 02:12:39 on dist-test-slave-6k22"
server_key: "e8b085873db7d7fed07278f9f90cc3ea"
server_key_iv: "8b62012774e492c037bcd8672fa8bd7b"
server_key_version: "encryptionkey@0"
I20260806 02:12:39.202306 21943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:39.217187 21943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:39.220460 21943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:39.220647 21943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:39.221206 21943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:39.221990 21943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:39.222066 21943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:39.222137 21943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:39.222172 21943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:39.231076 21943 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:35085
I20260806 02:12:39.231098 22075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:35085 every 8 connection(s)
I20260806 02:12:39.231994 21943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:12:39.240579 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 21943
I20260806 02:12:39.240731 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:12:39.241026 18713 external_mini_cluster.cc:1428] Setting key c29aafad179dfdd4fa5852d3d326e9c0
Aug 06 02:12:39 dist-test-slave-6k22 krb5kdc[21832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:39.245921 21920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:39.233776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:58627 (local address 127.18.70.126:45835)
0806 02:12:39.233906 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:39.233910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:39.234713 (+ 803us) server_negotiation.cc:407] Connection header received
0806 02:12:39.236239 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:39.236247 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:39.236295 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:39.236371 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:39.237492 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.238008 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.239095 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.239322 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.239400 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:39.241978 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:39.241981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:39.241983 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:39.242007 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:39.243841 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:39.244334 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:39.244340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:39.244345 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:39.244446 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:39.244814 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:39.244817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:39.244819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:39.245010 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:39.245121 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:39.245609 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 02:12:39.245750 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 02:12:39.246675 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:39.234131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45835 (local address 127.18.70.65:58627)
0806 02:12:39.234557 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:39.234598 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:39.236043 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:39.236532 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:39.236539 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:39.236912 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:39.237363 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:39.237377 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.238110 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.238112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:39.238988 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:39.238997 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.239451 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.239453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:39.239606 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:39.240133 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:39.240150 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:39.241811 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:39.243957 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:39.243961 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:39.243972 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:39.244232 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:39.244539 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:39.244542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:39.244543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:39.244708 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:39.245122 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:39.245130 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:39.245378 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 02:12:39.245569 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 02:12:39.245875 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":109,"threads_started":1}
I20260806 02:12:39.248081 22076 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45835
I20260806 02:12:39.248317 22076 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:39.248773 22076 heartbeater.cc:511] Master 127.18.70.126:45835 requested a full tablet report, sending...
I20260806 02:12:39.250463 21881 ts_manager.cc:194] Registered new tserver with Master: 403b6a55febf432eb3101faacdab43f6 (127.18.70.65:35085)
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
I20260806 02:12:39.252249 21881 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:58627
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:12:39.270073 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45835
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:39.378775 22081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:39.379019 22081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:39.379060 22081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:39.383028 22081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:39.383085 22081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:39.383179 22081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:12:39.388131 22081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:39.389068 22081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:39.390055 22081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:39.399641 22089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:39.399668 22086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:39.399690 22087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:39.399897 22081 server_base.cc:1034] running on GCE node
I20260806 02:12:39.400379 22081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:39.401078 22081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:39.402263 22081 hybrid_clock.cc:648] HybridClock initialized: now 1785982359402253 us; error 30 us; skew 500 ppm
Aug 06 02:12:39 dist-test-slave-6k22 krb5kdc[21832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:39.405067 22081 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:12:39.405975 22081 webserver.cc:460] Webserver started at http://127.18.70.66:45207/ using document root <none> and password file <none>
I20260806 02:12:39.406476 22081 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:39.406530 22081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:39.406685 22081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:39.408613 22081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "e67ba90cee28414982fb79d9d6b5abac"
format_stamp: "Formatted at 2026-08-06 02:12:39 on dist-test-slave-6k22"
server_key: "ce4dc77258c8cff4a94a9f5399a1807c"
server_key_iv: "73de9a4652c17feca07861dfc3bd813d"
server_key_version: "encryptionkey@0"
I20260806 02:12:39.409145 22081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "e67ba90cee28414982fb79d9d6b5abac"
format_stamp: "Formatted at 2026-08-06 02:12:39 on dist-test-slave-6k22"
server_key: "ce4dc77258c8cff4a94a9f5399a1807c"
server_key_iv: "73de9a4652c17feca07861dfc3bd813d"
server_key_version: "encryptionkey@0"
I20260806 02:12:39.412675 22081 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 02:12:39.414934 22096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:39.416280 22081 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 02:12:39.416399 22081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "e67ba90cee28414982fb79d9d6b5abac"
format_stamp: "Formatted at 2026-08-06 02:12:39 on dist-test-slave-6k22"
server_key: "ce4dc77258c8cff4a94a9f5399a1807c"
server_key_iv: "73de9a4652c17feca07861dfc3bd813d"
server_key_version: "encryptionkey@0"
I20260806 02:12:39.416496 22081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:39.429404 22081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:39.432492 22081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:39.432701 22081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:39.433272 22081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:39.434078 22081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:39.434149 22081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:39.434217 22081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:39.434264 22081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:39.443024 22081 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:42901
I20260806 02:12:39.443066 22213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:42901 every 8 connection(s)
I20260806 02:12:39.443897 22081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:12:39.446167 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 22081
I20260806 02:12:39.446318 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:12:39.446630 18713 external_mini_cluster.cc:1428] Setting key e467ed5872e2e5de8360b579b38baa56
Aug 06 02:12:39 dist-test-slave-6k22 krb5kdc[21832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
I20260806 02:12:39.458427 21920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:39.445642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:56549 (local address 127.18.70.126:45835)
0806 02:12:39.445788 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:39.445793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:39.446842 (+ 1049us) server_negotiation.cc:407] Connection header received
0806 02:12:39.448569 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:39.448572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:39.448621 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:39.448697 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:39.449783 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.450302 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.451525 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.451954 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.452110 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:39.454636 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:39.454642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:39.454647 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:39.454681 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:39.456414 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:39.456914 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:39.456920 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:39.456924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:39.457018 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:39.457355 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:39.457362 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:39.457365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:39.457606 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:39.457726 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:39.458180 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 02:12:39.458279 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 02:12:39.459265 22216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:39.446093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45835 (local address 127.18.70.66:56549)
0806 02:12:39.446653 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:39.446725 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:39.448370 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:39.448862 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:39.448869 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:39.449196 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:39.449645 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:39.449657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.450426 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.450430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:39.451372 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:39.451382 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.452129 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.452132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:39.452276 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:39.452871 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:39.452887 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:39.454466 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:39.456545 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:39.456550 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:39.456561 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:39.456806 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:39.457126 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:39.457128 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:39.457130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:39.457262 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:39.457709 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:39.457713 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:39.457966 (+ 253us) client_negotiation.cc:769] Sending connection context
0806 02:12:39.458114 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 02:12:39.458352 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":172,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
I20260806 02:12:39.460521 22214 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45835
I20260806 02:12:39.460767 22214 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:39.461232 22214 heartbeater.cc:511] Master 127.18.70.126:45835 requested a full tablet report, sending...
I20260806 02:12:39.462381 21881 ts_manager.cc:194] Registered new tserver with Master: e67ba90cee28414982fb79d9d6b5abac (127.18.70.66:42901)
I20260806 02:12:39.463178 21881 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:56549
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:12:39.478797 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45835
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:39.591416 22219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:39.591725 22219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:39.591773 22219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:39.595711 22219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:39.595758 22219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:39.595847 22219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:12:39.600936 22219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:39.601858 22219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:39.602785 22219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:39.612301 22225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:39.612318 22227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:39.612336 22224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:39.612555 22219 server_base.cc:1034] running on GCE node
I20260806 02:12:39.613021 22219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:39.613679 22219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:39.614866 22219 hybrid_clock.cc:648] HybridClock initialized: now 1785982359614845 us; error 39 us; skew 500 ppm
Aug 06 02:12:39 dist-test-slave-6k22 krb5kdc[21832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:39.617851 22219 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:12:39.618674 22219 webserver.cc:460] Webserver started at http://127.18.70.67:33163/ using document root <none> and password file <none>
I20260806 02:12:39.619300 22219 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:39.619392 22219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:39.619678 22219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:39.621668 22219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "e927f4bab5ee4ab7b383c4ab963cbace"
format_stamp: "Formatted at 2026-08-06 02:12:39 on dist-test-slave-6k22"
server_key: "60feef1db5770eb3f890d1fce1668234"
server_key_iv: "85405701901d306b3f36c161ebffbe69"
server_key_version: "encryptionkey@0"
I20260806 02:12:39.622234 22219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "e927f4bab5ee4ab7b383c4ab963cbace"
format_stamp: "Formatted at 2026-08-06 02:12:39 on dist-test-slave-6k22"
server_key: "60feef1db5770eb3f890d1fce1668234"
server_key_iv: "85405701901d306b3f36c161ebffbe69"
server_key_version: "encryptionkey@0"
I20260806 02:12:39.625831 22219 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:12:39.628157 22234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:39.629323 22219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 02:12:39.629460 22219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "e927f4bab5ee4ab7b383c4ab963cbace"
format_stamp: "Formatted at 2026-08-06 02:12:39 on dist-test-slave-6k22"
server_key: "60feef1db5770eb3f890d1fce1668234"
server_key_iv: "85405701901d306b3f36c161ebffbe69"
server_key_version: "encryptionkey@0"
I20260806 02:12:39.629565 22219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:39.648084 22219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:39.651022 22219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:39.651209 22219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:39.651932 22219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:39.652791 22219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:39.652860 22219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:39.652935 22219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:39.652978 22219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:39.661748 22219 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:43015
I20260806 02:12:39.661837 22351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:43015 every 8 connection(s)
I20260806 02:12:39.662645 22219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:12:39.665226 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 22219
I20260806 02:12:39.665349 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:12:39.665634 18713 external_mini_cluster.cc:1428] Setting key 4ad4c5379f5d2499d2bafbd6cb4ca81e
Aug 06 02:12:39 dist-test-slave-6k22 krb5kdc[21832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:39.676587 21920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:39.664579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:53883 (local address 127.18.70.126:45835)
0806 02:12:39.664733 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:39.664737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:39.665663 (+ 926us) server_negotiation.cc:407] Connection header received
0806 02:12:39.667255 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:39.667258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:39.667311 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:39.667411 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:39.668629 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.669174 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.670325 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.670558 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.670667 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:39.673240 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:39.673246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:39.673252 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:39.673275 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:39.674511 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:39.675012 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:39.675016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:39.675018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:39.675099 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:39.675435 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:39.675439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:39.675441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:39.675697 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:39.675807 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:39.676286 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 02:12:39.676403 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 02:12:39.677348 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:39.664939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45835 (local address 127.18.70.67:53883)
0806 02:12:39.665526 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:39.665567 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:39.667051 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:39.667621 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:39.667630 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:39.668013 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:39.668485 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:39.668498 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.669330 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.669333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:39.670198 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:39.670207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.670703 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.670707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:39.670884 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:39.671398 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:39.671414 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:39.673076 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:39.674630 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:39.674636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:39.674650 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:39.674905 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:39.675192 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:39.675195 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:39.675197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:39.675334 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:39.675805 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:39.675812 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:39.676056 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 02:12:39.676238 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 02:12:39.676550 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":145,"threads_started":1}
I20260806 02:12:39.678501 22352 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45835
I20260806 02:12:39.678742 22352 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:39.679215 22352 heartbeater.cc:511] Master 127.18.70.126:45835 requested a full tablet report, sending...
I20260806 02:12:39.680302 21881 ts_manager.cc:194] Registered new tserver with Master: e927f4bab5ee4ab7b383c4ab963cbace (127.18.70.67:43015)
I20260806 02:12:39.680828 21881 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:53883
I20260806 02:12:39.691000 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 02:12:39 dist-test-slave-6k22 krb5kdc[21832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982359, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 02:12:39 dist-test-slave-6k22 krb5kdc[21832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982359, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:39.716660 21920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:39.708948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45136 (local address 127.18.70.126:45835)
0806 02:12:39.709127 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:39.709130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:39.709200 (+ 70us) server_negotiation.cc:407] Connection header received
0806 02:12:39.709316 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:39.709318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:39.709360 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:39.709427 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:39.710051 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.710534 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.711297 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:39.711492 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:39.711618 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:39.713804 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:39.713807 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:39.713809 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:39.713831 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:39.715032 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:39.715408 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:39.715410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:39.715411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:39.715480 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:39.715793 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:39.715799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:39.715803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:39.716029 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:39.716130 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:39.716344 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 02:12:39.716431 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 02:12:39.717134 21881 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45136
I20260806 02:12:39.718026 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 21943
I20260806 02:12:39.725721 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 22081
I20260806 02:12:39.733785 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 22219
I20260806 02:12:39.740756 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 21846
2026-08-06T02:12:39Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2121 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:12:39 dist-test-slave-6k22 krb5kdc[22365](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:12:39 dist-test-slave-6k22 krb5kdc[22365](info): set up 2 sockets
Aug 06 02:12:39 dist-test-slave-6k22 krb5kdc[22365](info): commencing operation
krb5kdc: starting...
W20260806 02:12:40.853765 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:12:40 dist-test-slave-6k22 krb5kdc[22365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:12:40.979928 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:33271
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:33271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:41.098419 22378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:41.098690 22378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:41.098785 22378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:41.102742 22378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:41.102820 22378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:41.102859 22378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:41.102893 22378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:41.102923 22378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:41.107959 22378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:33271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:33271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:41.108932 22378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:41.109877 22378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:41.118096 22387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:41.118103 22385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:41.118103 22384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:41.118160 22378 server_base.cc:1034] running on GCE node
I20260806 02:12:41.118888 22378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:41.120028 22378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:41.121203 22378 hybrid_clock.cc:648] HybridClock initialized: now 1785982361121178 us; error 40 us; skew 500 ppm
Aug 06 02:12:41 dist-test-slave-6k22 krb5kdc[22365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982361, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:41.123826 22378 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:12:41.124617 22378 webserver.cc:460] Webserver started at http://127.18.70.126:44099/ using document root <none> and password file <none>
I20260806 02:12:41.125131 22378 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:41.125196 22378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:41.125344 22378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:41.127246 22378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "8ac93453b3e3423aa8863c679ac5fbb5"
format_stamp: "Formatted at 2026-08-06 02:12:41 on dist-test-slave-6k22"
server_key: "efe6a083875b92b359c17e84c7cf7637"
server_key_iv: "6870569fcea737a4e6ee8273abcd6445"
server_key_version: "encryptionkey@0"
I20260806 02:12:41.127799 22378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "8ac93453b3e3423aa8863c679ac5fbb5"
format_stamp: "Formatted at 2026-08-06 02:12:41 on dist-test-slave-6k22"
server_key: "efe6a083875b92b359c17e84c7cf7637"
server_key_iv: "6870569fcea737a4e6ee8273abcd6445"
server_key_version: "encryptionkey@0"
I20260806 02:12:41.131173 22378 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:12:41.133373 22394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:41.134536 22378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 02:12:41.134639 22378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "8ac93453b3e3423aa8863c679ac5fbb5"
format_stamp: "Formatted at 2026-08-06 02:12:41 on dist-test-slave-6k22"
server_key: "efe6a083875b92b359c17e84c7cf7637"
server_key_iv: "6870569fcea737a4e6ee8273abcd6445"
server_key_version: "encryptionkey@0"
I20260806 02:12:41.134718 22378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:41.151858 22378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:41.155705 22378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:41.155905 22378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:41.162571 22378 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:33271
I20260806 02:12:41.162707 22446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:33271 every 8 connection(s)
I20260806 02:12:41.163524 22378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:41.166235 22447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:41.166472 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 22378
I20260806 02:12:41.166606 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:41.166909 18713 external_mini_cluster.cc:1428] Setting key c5cc8aa9ad71b89973eb54aeede55c1d
I20260806 02:12:41.173177 22447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5: Bootstrap starting.
I20260806 02:12:41.175758 22447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5: Neither blocks nor log segments found. Creating new log.
Aug 06 02:12:41 dist-test-slave-6k22 krb5kdc[22365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:41.176517 22447 log.cc:824] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:41.178788 22447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5: No bootstrap required, opened a new log
I20260806 02:12:41.182227 22450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:41.168376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33990 (local address 127.18.70.126:33271)
0806 02:12:41.168775 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:41.168793 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:41.168835 (+ 42us) server_negotiation.cc:407] Connection header received
0806 02:12:41.169566 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:41.169590 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:41.169993 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:41.170360 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:41.171138 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.172586 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.173559 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.174237 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.174405 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:41.177053 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:41.177058 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:41.177070 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:41.177094 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:41.179039 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:41.179498 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:41.179505 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:41.179510 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:41.179680 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:41.179994 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:41.179998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:41.179999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:41.180458 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:41.180659 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:41.180965 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 02:12:41.181308 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":105,"threads_started":1}
I20260806 02:12:41.183279 22447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac93453b3e3423aa8863c679ac5fbb5" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33271 } }
I20260806 02:12:41.183552 22447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:41.183624 22447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ac93453b3e3423aa8863c679ac5fbb5, State: Initialized, Role: FOLLOWER
I20260806 02:12:41.184060 22447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac93453b3e3423aa8863c679ac5fbb5" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33271 } }
I20260806 02:12:41.184216 22447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:41.184300 22447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:41.184444 22447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:41.185292 22447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac93453b3e3423aa8863c679ac5fbb5" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33271 } }
I20260806 02:12:41.185612 22447 leader_election.cc:302] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ac93453b3e3423aa8863c679ac5fbb5; no voters:
I20260806 02:12:41.185889 22447 leader_election.cc:288] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:41.186007 22452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:41.186275 22452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [term 1 LEADER]: Becoming Leader. State: Replica: 8ac93453b3e3423aa8863c679ac5fbb5, State: Running, Role: LEADER
I20260806 02:12:41.186580 22452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac93453b3e3423aa8863c679ac5fbb5" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33271 } }
I20260806 02:12:41.187027 22447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:41.188220 22454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ac93453b3e3423aa8863c679ac5fbb5. Latest consensus state: current_term: 1 leader_uuid: "8ac93453b3e3423aa8863c679ac5fbb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac93453b3e3423aa8863c679ac5fbb5" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33271 } } }
I20260806 02:12:41.188354 22454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:41.188575 22453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ac93453b3e3423aa8863c679ac5fbb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ac93453b3e3423aa8863c679ac5fbb5" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33271 } } }
I20260806 02:12:41.188714 22453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:41.188805 22461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:41.192072 22461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:41.197451 22461 catalog_manager.cc:1347] Generated new cluster ID: 8642865486e64c988851897d72ba4426
I20260806 02:12:41.197525 22461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:41.208446 22461 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:41.209270 22461 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:41.218506 22461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ac93453b3e3423aa8863c679ac5fbb5: Generated new TSK 0
I20260806 02:12:41.219105 22461 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:12:41.249765 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33271
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:41.368100 22474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:41.368343 22474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:41.368382 22474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:41.372237 22474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:41.372289 22474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:41.372344 22474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:12:41.377383 22474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:41.378378 22474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:41.379329 22474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:41.388638 22482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:41.388685 22480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:41.388684 22479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:41.388851 22474 server_base.cc:1034] running on GCE node
I20260806 02:12:41.389310 22474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:41.390002 22474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:41.391192 22474 hybrid_clock.cc:648] HybridClock initialized: now 1785982361391169 us; error 48 us; skew 500 ppm
Aug 06 02:12:41 dist-test-slave-6k22 krb5kdc[22365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982361, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:41.394017 22474 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:12:41.395291 22474 webserver.cc:460] Webserver started at http://127.18.70.65:39337/ using document root <none> and password file <none>
I20260806 02:12:41.395955 22474 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:41.396049 22474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:41.396267 22474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:41.398340 22474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "ea77fc0a11ad4f91949ef9b472802901"
format_stamp: "Formatted at 2026-08-06 02:12:41 on dist-test-slave-6k22"
server_key: "18d5ae250a03429605a23b9bca55941b"
server_key_iv: "5c414bb2783b2da8a4101d0b75a62681"
server_key_version: "encryptionkey@0"
I20260806 02:12:41.398937 22474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "ea77fc0a11ad4f91949ef9b472802901"
format_stamp: "Formatted at 2026-08-06 02:12:41 on dist-test-slave-6k22"
server_key: "18d5ae250a03429605a23b9bca55941b"
server_key_iv: "5c414bb2783b2da8a4101d0b75a62681"
server_key_version: "encryptionkey@0"
I20260806 02:12:41.402738 22474 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 02:12:41.405201 22489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:41.406497 22474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:41.406626 22474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "ea77fc0a11ad4f91949ef9b472802901"
format_stamp: "Formatted at 2026-08-06 02:12:41 on dist-test-slave-6k22"
server_key: "18d5ae250a03429605a23b9bca55941b"
server_key_iv: "5c414bb2783b2da8a4101d0b75a62681"
server_key_version: "encryptionkey@0"
I20260806 02:12:41.406742 22474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:41.416926 22474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:41.420176 22474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:41.420368 22474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:41.420925 22474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:41.421808 22474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:41.421878 22474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:41.421945 22474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:41.421975 22474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:41.430963 22474 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:43575
I20260806 02:12:41.430994 22606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:43575 every 8 connection(s)
I20260806 02:12:41.431869 22474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:12:41.436305 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 22474
I20260806 02:12:41.436427 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:12:41.436750 18713 external_mini_cluster.cc:1428] Setting key 32ff840f202968bc2f8811b1e07fbe31
Aug 06 02:12:41 dist-test-slave-6k22 krb5kdc[22365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982361, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:41.445847 22450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:41.433603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:58089 (local address 127.18.70.126:33271)
0806 02:12:41.433778 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:41.433783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:41.434588 (+ 805us) server_negotiation.cc:407] Connection header received
0806 02:12:41.435941 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:41.435944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:41.435989 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:41.436065 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:41.437548 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.438167 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.439391 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.439736 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.439849 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:41.442272 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:41.442275 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:41.442278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:41.442303 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:41.443772 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:41.444289 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:41.444294 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:41.444296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:41.444382 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:41.444726 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:41.444732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:41.444737 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:41.444927 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:41.445035 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:41.445528 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 02:12:41.445654 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 02:12:41.446604 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:41.433992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:33271 (local address 127.18.70.65:58089)
0806 02:12:41.434434 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:41.434476 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:41.435738 (+ 1262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:41.436288 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:41.436297 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:41.436785 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:41.437414 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:41.437436 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.438320 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.438323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:41.439265 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:41.439274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.439889 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.439893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:41.440053 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:41.440665 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:41.440682 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:41.442108 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:41.443884 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:41.443890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:41.443908 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:41.444183 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:41.444484 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:41.444488 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:41.444490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:41.444629 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:41.445047 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:41.445054 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:41.445301 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 02:12:41.445486 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 02:12:41.445804 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
I20260806 02:12:41.447916 22607 heartbeater.cc:347] Connected to a master server at 127.18.70.126:33271
I20260806 02:12:41.448170 22607 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:41.448709 22607 heartbeater.cc:511] Master 127.18.70.126:33271 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
I20260806 02:12:41.450450 22411 ts_manager.cc:194] Registered new tserver with Master: ea77fc0a11ad4f91949ef9b472802901 (127.18.70.65:43575)
I20260806 02:12:41.451818 22411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:58089
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:12:41.466415 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33271
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:41.578115 22612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:41.578408 22612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:41.578454 22612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:41.582540 22612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:41.582593 22612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:41.582692 22612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:12:41.588555 22612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:41.589648 22612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:41.590677 22612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:41.600031 22617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:41.600039 22618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:41.600088 22620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:41.600435 22612 server_base.cc:1034] running on GCE node
I20260806 02:12:41.600869 22612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:41.601460 22612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:41.602645 22612 hybrid_clock.cc:648] HybridClock initialized: now 1785982361602611 us; error 63 us; skew 500 ppm
Aug 06 02:12:41 dist-test-slave-6k22 krb5kdc[22365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982361, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:41.605429 22612 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:12:41.606258 22612 webserver.cc:460] Webserver started at http://127.18.70.66:33843/ using document root <none> and password file <none>
I20260806 02:12:41.606796 22612 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:41.606900 22612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:41.607146 22612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:41.609259 22612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "a600b8e4e27d464a87f81e7b4cdc7a30"
format_stamp: "Formatted at 2026-08-06 02:12:41 on dist-test-slave-6k22"
server_key: "8bcfa096c315ae2d6e1484fc5791b5ab"
server_key_iv: "e10dbda601db86a804e22a175d1de962"
server_key_version: "encryptionkey@0"
I20260806 02:12:41.609815 22612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "a600b8e4e27d464a87f81e7b4cdc7a30"
format_stamp: "Formatted at 2026-08-06 02:12:41 on dist-test-slave-6k22"
server_key: "8bcfa096c315ae2d6e1484fc5791b5ab"
server_key_iv: "e10dbda601db86a804e22a175d1de962"
server_key_version: "encryptionkey@0"
I20260806 02:12:41.613546 22612 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 02:12:41.616039 22627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:41.617268 22612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:41.617417 22612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "a600b8e4e27d464a87f81e7b4cdc7a30"
format_stamp: "Formatted at 2026-08-06 02:12:41 on dist-test-slave-6k22"
server_key: "8bcfa096c315ae2d6e1484fc5791b5ab"
server_key_iv: "e10dbda601db86a804e22a175d1de962"
server_key_version: "encryptionkey@0"
I20260806 02:12:41.617553 22612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:41.634761 22612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:41.638042 22612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:41.638231 22612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:41.638880 22612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:41.639799 22612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:41.639868 22612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:41.639940 22612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:41.639987 22612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:41.648991 22612 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:34241
I20260806 02:12:41.649030 22743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:34241 every 8 connection(s)
I20260806 02:12:41.649878 22612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:12:41.652639 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 22612
I20260806 02:12:41.652760 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:12:41.653053 18713 external_mini_cluster.cc:1428] Setting key a1e58abce93f8407443eaed67dbb9f81
Aug 06 02:12:41 dist-test-slave-6k22 krb5kdc[22365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982361, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
I20260806 02:12:41.665366 22450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:41.651631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:54383 (local address 127.18.70.126:33271)
0806 02:12:41.651771 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:41.651775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:41.652624 (+ 849us) server_negotiation.cc:407] Connection header received
0806 02:12:41.654344 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:41.654349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:41.654408 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:41.654488 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:41.655693 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.656533 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.658337 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.658560 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.658639 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:41.661495 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:41.661501 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:41.661507 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:41.661543 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:41.663243 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:41.663833 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:41.663838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:41.663841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:41.663946 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:41.664302 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:41.664305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:41.664307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:41.664571 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:41.664683 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:41.665140 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 02:12:41.665249 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:12:41.666170 22746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:41.652021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:33271 (local address 127.18.70.66:54383)
0806 02:12:41.652479 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:41.652521 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:41.654092 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:41.654672 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:41.654680 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:41.655022 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:41.655505 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:41.655519 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.656685 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.656691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:41.658196 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:41.658212 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.658724 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.658731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:41.658965 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:41.659603 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:41.659632 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:41.661314 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:41.663354 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:41.663359 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:41.663370 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:41.663708 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:41.664057 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:41.664059 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:41.664061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:41.664192 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:41.664676 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:41.664681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:41.664908 (+ 227us) client_negotiation.cc:769] Sending connection context
0806 02:12:41.665056 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 02:12:41.665315 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":117,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
I20260806 02:12:41.667387 22744 heartbeater.cc:347] Connected to a master server at 127.18.70.126:33271
I20260806 02:12:41.667672 22744 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:41.668159 22744 heartbeater.cc:511] Master 127.18.70.126:33271 requested a full tablet report, sending...
I20260806 02:12:41.669279 22411 ts_manager.cc:194] Registered new tserver with Master: a600b8e4e27d464a87f81e7b4cdc7a30 (127.18.70.66:34241)
I20260806 02:12:41.669860 22411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:54383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:12:41.682541 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33271
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:46593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:41.796048 22749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:41.796339 22749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:41.796442 22749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:41.800501 22749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:41.800573 22749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:41.800653 22749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:12:41.805720 22749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:46593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:41.806766 22749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:41.807873 22749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:41.817373 22755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:41.817404 22754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:41.817481 22757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:41.817767 22749 server_base.cc:1034] running on GCE node
I20260806 02:12:41.818086 22749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:41.818647 22749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:41.819803 22749 hybrid_clock.cc:648] HybridClock initialized: now 1785982361819784 us; error 31 us; skew 500 ppm
Aug 06 02:12:41 dist-test-slave-6k22 krb5kdc[22365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982361, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:41.822455 22749 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:12:41.823392 22749 webserver.cc:460] Webserver started at http://127.18.70.67:42211/ using document root <none> and password file <none>
I20260806 02:12:41.824018 22749 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:41.824111 22749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:41.824332 22749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:41.826390 22749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "45f5f4da4602479ca405a4dd5937d81c"
format_stamp: "Formatted at 2026-08-06 02:12:41 on dist-test-slave-6k22"
server_key: "281e72ab78fef23ea0f00fb544c5d807"
server_key_iv: "ec6b998212beb8cab8cf1841cbb0bd35"
server_key_version: "encryptionkey@0"
I20260806 02:12:41.826963 22749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "45f5f4da4602479ca405a4dd5937d81c"
format_stamp: "Formatted at 2026-08-06 02:12:41 on dist-test-slave-6k22"
server_key: "281e72ab78fef23ea0f00fb544c5d807"
server_key_iv: "ec6b998212beb8cab8cf1841cbb0bd35"
server_key_version: "encryptionkey@0"
I20260806 02:12:41.830652 22749 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 02:12:41.833014 22764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:41.834146 22749 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:41.834286 22749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "45f5f4da4602479ca405a4dd5937d81c"
format_stamp: "Formatted at 2026-08-06 02:12:41 on dist-test-slave-6k22"
server_key: "281e72ab78fef23ea0f00fb544c5d807"
server_key_iv: "ec6b998212beb8cab8cf1841cbb0bd35"
server_key_version: "encryptionkey@0"
I20260806 02:12:41.834393 22749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:41.861414 22749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:41.864355 22749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:41.864571 22749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:41.865190 22749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:41.866017 22749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:41.866087 22749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:41.866159 22749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:41.866211 22749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:41.875682 22749 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:42617
I20260806 02:12:41.875728 22879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:42617 every 8 connection(s)
I20260806 02:12:41.876619 22749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:12:41.878937 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 22749
I20260806 02:12:41.879072 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:12:41.879350 18713 external_mini_cluster.cc:1428] Setting key 0234588152d4d8148ada259f6eeff22d
Aug 06 02:12:41 dist-test-slave-6k22 krb5kdc[22365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982361, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:41.890271 22450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:41.878451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:42905 (local address 127.18.70.126:33271)
0806 02:12:41.878581 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:41.878592 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:41.879456 (+ 864us) server_negotiation.cc:407] Connection header received
0806 02:12:41.881278 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:41.881282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:41.881335 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:41.881421 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:41.882657 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.883189 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.884360 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.884569 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.884648 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:41.887101 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:41.887104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:41.887106 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:41.887129 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:41.888483 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:41.888928 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:41.888930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:41.888932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:41.889001 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:41.889356 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:41.889358 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:41.889359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:41.889523 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:41.889605 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:41.890053 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 02:12:41.890146 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 02:12:41.891062 22883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:41.878778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:33271 (local address 127.18.70.67:42905)
0806 02:12:41.879298 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:41.879352 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:41.881074 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:41.881636 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:41.881643 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:41.881997 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:41.882532 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:41.882546 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.883309 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.883311 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:41.884254 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:41.884264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.884696 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.884699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:41.884838 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:41.885482 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:41.885498 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:41.886960 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:41.888583 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:41.888588 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:41.888599 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:41.888839 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:41.889120 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:41.889122 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:41.889123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:41.889275 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:41.889612 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:41.889617 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:41.889852 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 02:12:41.890017 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 02:12:41.890262 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":142,"threads_started":1}
I20260806 02:12:41.892409 22881 heartbeater.cc:347] Connected to a master server at 127.18.70.126:33271
I20260806 02:12:41.892676 22881 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:41.893277 22881 heartbeater.cc:511] Master 127.18.70.126:33271 requested a full tablet report, sending...
I20260806 02:12:41.894340 22411 ts_manager.cc:194] Registered new tserver with Master: 45f5f4da4602479ca405a4dd5937d81c (127.18.70.67:42617)
I20260806 02:12:41.894848 22411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:42905
I20260806 02:12:41.904918 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 02:12:41 dist-test-slave-6k22 krb5kdc[22365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 02:12:41 dist-test-slave-6k22 krb5kdc[22365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.65@KRBTEST.COM
I20260806 02:12:41.932407 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:41.923336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52486 (local address 127.18.70.65:43575)
0806 02:12:41.923646 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:41.923664 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:41.923697 (+ 33us) server_negotiation.cc:407] Connection header received
0806 02:12:41.923751 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:41.923756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:41.923950 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:41.924099 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:41.924748 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.925519 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.926366 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.926736 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.926818 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:41.929187 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:41.929196 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:41.929209 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:41.929233 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:41.930859 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:41.931211 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:41.931216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:41.931217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:41.931304 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:41.931524 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:41.931526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:41.931527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:41.931798 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:41.931993 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:41.932128 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 02:12:41.932259 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":95,"threads_started":1}
Aug 06 02:12:41 dist-test-slave-6k22 krb5kdc[22365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:41.942598 22450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:41.935275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33996 (local address 127.18.70.126:33271)
0806 02:12:41.935413 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:41.935417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:41.935431 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:12:41.935520 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:41.935522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:41.935604 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:41.935673 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:41.936164 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.936643 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.937331 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:41.937521 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:41.937593 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:41.940011 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:41.940014 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:41.940016 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:41.940038 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:41.941170 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:41.941519 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:41.941521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:41.941522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:41.941590 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:41.941804 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:41.941806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:41.941807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:41.941993 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:41.942089 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:41.942291 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 02:12:41.942378 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 02:12:41.943010 22411 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33996
I20260806 02:12:41.943549 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 22474
I20260806 02:12:41.951654 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 22612
I20260806 02:12:41.958200 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 22749
I20260806 02:12:41.964589 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 22378
2026-08-06T02:12:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2224 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:12:42 dist-test-slave-6k22 krb5kdc[22893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:12:42 dist-test-slave-6k22 krb5kdc[22893](info): set up 2 sockets
Aug 06 02:12:42 dist-test-slave-6k22 krb5kdc[22893](info): commencing operation
krb5kdc: starting...
W20260806 02:12:43.079334 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:12:43 dist-test-slave-6k22 krb5kdc[22893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:12:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:12:43.196151 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:41661
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:41661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:43.306511 22907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:43.306739 22907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:43.306785 22907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:43.310634 22907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:43.310685 22907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:43.310705 22907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:43.310721 22907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:43.310736 22907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:43.315912 22907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:41661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:41661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:43.316807 22907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:43.317680 22907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:43.325812 22912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:43.325839 22915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:43.325799 22913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:43.326076 22907 server_base.cc:1034] running on GCE node
I20260806 02:12:43.326731 22907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:43.327880 22907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:43.329038 22907 hybrid_clock.cc:648] HybridClock initialized: now 1785982363329016 us; error 36 us; skew 500 ppm
Aug 06 02:12:43 dist-test-slave-6k22 krb5kdc[22893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982363, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:43.335927 22907 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:12:43.336686 22907 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:43.336795 22907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:43.337049 22907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:43.339061 22907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "9626bca2305d4912b5544aae0d1e5169"
format_stamp: "Formatted at 2026-08-06 02:12:43 on dist-test-slave-6k22"
server_key: "8477875dcbe6735e78427f6411fa3934"
server_key_iv: "df5ba0fc660a6e14fe1e74a69f849502"
server_key_version: "encryptionkey@0"
I20260806 02:12:43.339684 22907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "9626bca2305d4912b5544aae0d1e5169"
format_stamp: "Formatted at 2026-08-06 02:12:43 on dist-test-slave-6k22"
server_key: "8477875dcbe6735e78427f6411fa3934"
server_key_iv: "df5ba0fc660a6e14fe1e74a69f849502"
server_key_version: "encryptionkey@0"
I20260806 02:12:43.343344 22907 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 02:12:43.345701 22921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:43.346884 22907 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:12:43.347019 22907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "9626bca2305d4912b5544aae0d1e5169"
format_stamp: "Formatted at 2026-08-06 02:12:43 on dist-test-slave-6k22"
server_key: "8477875dcbe6735e78427f6411fa3934"
server_key_iv: "df5ba0fc660a6e14fe1e74a69f849502"
server_key_version: "encryptionkey@0"
I20260806 02:12:43.347126 22907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:43.379465 22907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:43.382741 22907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:43.382969 22907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:43.390225 22907 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:41661
I20260806 02:12:43.390468 22976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:41661 every 8 connection(s)
I20260806 02:12:43.390900 22907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:43.392216 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 22907
I20260806 02:12:43.392383 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:43.392650 18713 external_mini_cluster.cc:1428] Setting key ae5dad77e1cc59745268554e3bd0131e
I20260806 02:12:43.394611 22977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:43.401291 22977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169: Bootstrap starting.
Aug 06 02:12:43 dist-test-slave-6k22 krb5kdc[22893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:43.403908 22977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:43.404693 22977 log.cc:824] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:43.406695 22977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169: No bootstrap required, opened a new log
I20260806 02:12:43.407361 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:43.394038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36392 (local address 127.18.70.126:41661)
0806 02:12:43.394484 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:43.394503 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:43.394546 (+ 43us) server_negotiation.cc:407] Connection header received
0806 02:12:43.395405 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:43.395428 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:43.395895 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:43.396243 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:43.396864 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:43.398379 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:43.399218 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:43.399930 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:43.400078 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:43.402711 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:43.402716 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:43.402728 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:43.402756 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:43.404563 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:43.405040 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:43.405045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:43.405049 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:43.405156 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:43.405459 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:43.405461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:43.405462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:43.405860 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:43.406062 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:43.406379 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 02:12:43.406638 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":122,"threads_started":1}
I20260806 02:12:43.410063 22977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9626bca2305d4912b5544aae0d1e5169" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 41661 } }
I20260806 02:12:43.410260 22977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:43.410337 22977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9626bca2305d4912b5544aae0d1e5169, State: Initialized, Role: FOLLOWER
I20260806 02:12:43.410818 22977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9626bca2305d4912b5544aae0d1e5169" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 41661 } }
I20260806 02:12:43.410969 22977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:43.411033 22977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:43.411181 22977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:43.412084 22977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9626bca2305d4912b5544aae0d1e5169" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 41661 } }
I20260806 02:12:43.412438 22977 leader_election.cc:302] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9626bca2305d4912b5544aae0d1e5169; no voters:
I20260806 02:12:43.412740 22977 leader_election.cc:288] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:43.412864 22982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:43.413097 22982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [term 1 LEADER]: Becoming Leader. State: Replica: 9626bca2305d4912b5544aae0d1e5169, State: Running, Role: LEADER
I20260806 02:12:43.413455 22982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9626bca2305d4912b5544aae0d1e5169" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 41661 } }
I20260806 02:12:43.413867 22977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:43.414978 22984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9626bca2305d4912b5544aae0d1e5169. Latest consensus state: current_term: 1 leader_uuid: "9626bca2305d4912b5544aae0d1e5169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9626bca2305d4912b5544aae0d1e5169" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 41661 } } }
I20260806 02:12:43.415112 22984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:43.415454 22983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9626bca2305d4912b5544aae0d1e5169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9626bca2305d4912b5544aae0d1e5169" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 41661 } } }
I20260806 02:12:43.415526 22991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:43.415668 22983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:43.418083 22991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:43.423581 22991 catalog_manager.cc:1347] Generated new cluster ID: 3c2132fd2fd74b83877008858e66f8c0
I20260806 02:12:43.423642 22991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:43.431154 22991 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:43.432380 22991 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:43.441962 22991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169: Generated new TSK 0
I20260806 02:12:43.442700 22991 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:12:43.477883 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:41661
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:43.592931 23003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:43.593181 23003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:43.593267 23003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:43.597282 23003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:43.597338 23003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:43.597458 23003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:12:43.602577 23003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:41661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:43.603648 23003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:43.604674 23003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:43.613646 23009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:43.613646 23011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:43.613735 23008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:43.614003 23003 server_base.cc:1034] running on GCE node
I20260806 02:12:43.614512 23003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:43.615311 23003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:43.616482 23003 hybrid_clock.cc:648] HybridClock initialized: now 1785982363616443 us; error 54 us; skew 500 ppm
Aug 06 02:12:43 dist-test-slave-6k22 krb5kdc[22893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982363, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:43.619203 23003 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:12:43.619963 23003 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:43.620049 23003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:43.620265 23003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:43.622157 23003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "598b9d2666ff453599e083503a20e9d5"
format_stamp: "Formatted at 2026-08-06 02:12:43 on dist-test-slave-6k22"
server_key: "e63c294e116321d862aa3e3922b77138"
server_key_iv: "a61cbb0540075be7aa35e6405c0c7182"
server_key_version: "encryptionkey@0"
I20260806 02:12:43.622704 23003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "598b9d2666ff453599e083503a20e9d5"
format_stamp: "Formatted at 2026-08-06 02:12:43 on dist-test-slave-6k22"
server_key: "e63c294e116321d862aa3e3922b77138"
server_key_iv: "a61cbb0540075be7aa35e6405c0c7182"
server_key_version: "encryptionkey@0"
I20260806 02:12:43.626246 23003 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 02:12:43.628891 23017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:43.630147 23003 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:43.630295 23003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "598b9d2666ff453599e083503a20e9d5"
format_stamp: "Formatted at 2026-08-06 02:12:43 on dist-test-slave-6k22"
server_key: "e63c294e116321d862aa3e3922b77138"
server_key_iv: "a61cbb0540075be7aa35e6405c0c7182"
server_key_version: "encryptionkey@0"
I20260806 02:12:43.630406 23003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:43.640362 23003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:43.643357 23003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:43.643597 23003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:43.644176 23003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:43.644996 23003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:43.645066 23003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:43.645133 23003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:43.645180 23003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:43.654417 23003 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:36993
I20260806 02:12:43.654439 23137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:36993 every 8 connection(s)
I20260806 02:12:43.655063 23003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:12:43.664309 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 23003
I20260806 02:12:43.664455 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
Aug 06 02:12:43 dist-test-slave-6k22 krb5kdc[22893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982363, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:43.664850 18713 external_mini_cluster.cc:1428] Setting key cc1603643b490bf248801413089d5b12
I20260806 02:12:43.669067 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:43.656852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:53001 (local address 127.18.70.126:41661)
0806 02:12:43.656985 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:43.656988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:43.657766 (+ 778us) server_negotiation.cc:407] Connection header received
0806 02:12:43.659115 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:43.659118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:43.659180 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:43.659298 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:43.660508 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:43.661054 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:43.662210 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:43.662457 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:43.662573 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:43.665438 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:43.665442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:43.665444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:43.665472 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:43.667020 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:43.667581 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:43.667586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:43.667588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:43.667676 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:43.668017 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:43.668022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:43.668024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:43.668206 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:43.668310 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:43.668782 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 02:12:43.668903 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:12:43.669888 23141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:43.657183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:41661 (local address 127.18.70.65:53001)
0806 02:12:43.657602 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:43.657645 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:43.658889 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:43.659479 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:43.659486 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:43.659874 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:43.660349 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:43.660362 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:43.661185 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:43.661191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:43.662067 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:43.662076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:43.662618 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:43.662622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:43.662821 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:43.663392 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:43.663407 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:43.665257 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:43.667170 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:43.667176 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:43.667190 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:43.667441 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:43.667771 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:43.667775 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:43.667777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:43.667920 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:43.668310 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:43.668318 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:43.668556 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 02:12:43.668742 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 02:12:43.669052 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":103,"threads_started":1}
I20260806 02:12:43.671222 23139 heartbeater.cc:347] Connected to a master server at 127.18.70.126:41661
I20260806 02:12:43.671495 23139 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:43.672094 23139 heartbeater.cc:511] Master 127.18.70.126:41661 requested a full tablet report, sending...
I20260806 02:12:43.674026 22938 ts_manager.cc:194] Registered new tserver with Master: 598b9d2666ff453599e083503a20e9d5 (127.18.70.65:36993)
I20260806 02:12:43.675731 22938 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:53001
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:12:43.694533 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:41661
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:43.806927 23144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:43.807163 23144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:43.807204 23144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:43.811077 23144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:43.811128 23144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:43.811228 23144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:12:43.816447 23144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:41661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:43.817410 23144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:43.818416 23144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:43.827515 23150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:43.827601 23144 server_base.cc:1034] running on GCE node
W20260806 02:12:43.827605 23149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:43.827515 23152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:43.828099 23144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:43.828752 23144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:43.829952 23144 hybrid_clock.cc:648] HybridClock initialized: now 1785982363829915 us; error 47 us; skew 500 ppm
Aug 06 02:12:43 dist-test-slave-6k22 krb5kdc[22893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982363, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:43.832679 23144 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:12:43.833485 23144 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:43.833580 23144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:43.833811 23144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:43.835950 23144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "5afc524267214d8cac7489369a8f2c65"
format_stamp: "Formatted at 2026-08-06 02:12:43 on dist-test-slave-6k22"
server_key: "dd7908f0565e8ac645956fd0c70afeb8"
server_key_iv: "7e2c65143c9c82bbabc4e2a9d1eef154"
server_key_version: "encryptionkey@0"
I20260806 02:12:43.836501 23144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "5afc524267214d8cac7489369a8f2c65"
format_stamp: "Formatted at 2026-08-06 02:12:43 on dist-test-slave-6k22"
server_key: "dd7908f0565e8ac645956fd0c70afeb8"
server_key_iv: "7e2c65143c9c82bbabc4e2a9d1eef154"
server_key_version: "encryptionkey@0"
I20260806 02:12:43.840281 23144 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 02:12:43.842720 23159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:43.844034 23144 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:43.844189 23144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "5afc524267214d8cac7489369a8f2c65"
format_stamp: "Formatted at 2026-08-06 02:12:43 on dist-test-slave-6k22"
server_key: "dd7908f0565e8ac645956fd0c70afeb8"
server_key_iv: "7e2c65143c9c82bbabc4e2a9d1eef154"
server_key_version: "encryptionkey@0"
I20260806 02:12:43.844298 23144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:43.857669 23144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:43.860495 23144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:43.860718 23144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:43.861281 23144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:43.862071 23144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:43.862147 23144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:43.862217 23144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:43.862264 23144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:43.871227 23144 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:36285
I20260806 02:12:43.871248 23277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:36285 every 8 connection(s)
I20260806 02:12:43.871929 23144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:12:43.881093 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 23144
Aug 06 02:12:43 dist-test-slave-6k22 krb5kdc[22893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982363, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:43.881220 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:12:43.881517 18713 external_mini_cluster.cc:1428] Setting key f75322da7c74a0ec6fbf45faed20d492
I20260806 02:12:43.885720 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:43.873703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:49357 (local address 127.18.70.126:41661)
0806 02:12:43.873843 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:43.873847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:43.874624 (+ 777us) server_negotiation.cc:407] Connection header received
0806 02:12:43.875992 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:43.875996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:43.876054 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:43.876129 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:43.877282 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:43.877839 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:43.878992 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:43.879242 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:43.879366 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:43.881813 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:43.881817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:43.881820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:43.881849 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:43.883541 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:43.884068 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:43.884075 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:43.884079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:43.884174 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:43.884490 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:43.884495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:43.884500 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:43.884745 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:43.884859 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:43.885420 (+ 561us) server_negotiation.cc:299] Negotiation successful
0806 02:12:43.885535 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:12:43.886539 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:43.874014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:41661 (local address 127.18.70.66:49357)
0806 02:12:43.874475 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:43.874518 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:43.875781 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:43.876289 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:43.876296 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:43.876658 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:43.877124 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:43.877137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:43.877980 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:43.877983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:43.878863 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:43.878872 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:43.879408 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:43.879412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:43.879641 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:43.880138 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:43.880154 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:43.881617 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:43.883693 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:43.883698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:43.883709 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:43.883961 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:43.884268 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:43.884270 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:43.884271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:43.884398 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:43.884857 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:43.884862 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:43.885126 (+ 264us) client_negotiation.cc:769] Sending connection context
0806 02:12:43.885361 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 02:12:43.885684 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":115,"threads_started":1}
I20260806 02:12:43.887713 23278 heartbeater.cc:347] Connected to a master server at 127.18.70.126:41661
I20260806 02:12:43.887960 23278 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:43.888397 23278 heartbeater.cc:511] Master 127.18.70.126:41661 requested a full tablet report, sending...
I20260806 02:12:43.889461 22938 ts_manager.cc:194] Registered new tserver with Master: 5afc524267214d8cac7489369a8f2c65 (127.18.70.66:36285)
I20260806 02:12:43.890252 22938 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:49357
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:12:43.909904 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:41661
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:44.022328 23284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:44.022562 23284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:44.022605 23284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:44.026731 23284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:44.026790 23284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:44.026865 23284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:12:44.031931 23284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:41661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:44.032872 23284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:44.033944 23284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:44.043591 23290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:44.043588 23289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:44.043613 23292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:44.043817 23284 server_base.cc:1034] running on GCE node
I20260806 02:12:44.044281 23284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:44.044922 23284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:44.046136 23284 hybrid_clock.cc:648] HybridClock initialized: now 1785982364046117 us; error 26 us; skew 500 ppm
Aug 06 02:12:44 dist-test-slave-6k22 krb5kdc[22893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982364, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:44.049145 23284 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:12:44.049983 23284 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:44.050083 23284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:44.050381 23284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:44.052722 23284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "58495175fa6d45038bbdf906d5ae3ae5"
format_stamp: "Formatted at 2026-08-06 02:12:44 on dist-test-slave-6k22"
server_key: "a3d0b7990e34b9f10bee0e47d08ba781"
server_key_iv: "ceff412e5d6a08a7445badbc6a1ac91f"
server_key_version: "encryptionkey@0"
I20260806 02:12:44.053320 23284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "58495175fa6d45038bbdf906d5ae3ae5"
format_stamp: "Formatted at 2026-08-06 02:12:44 on dist-test-slave-6k22"
server_key: "a3d0b7990e34b9f10bee0e47d08ba781"
server_key_iv: "ceff412e5d6a08a7445badbc6a1ac91f"
server_key_version: "encryptionkey@0"
I20260806 02:12:44.057214 23284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 02:12:44.060158 23299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:44.061406 23284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:44.061549 23284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "58495175fa6d45038bbdf906d5ae3ae5"
format_stamp: "Formatted at 2026-08-06 02:12:44 on dist-test-slave-6k22"
server_key: "a3d0b7990e34b9f10bee0e47d08ba781"
server_key_iv: "ceff412e5d6a08a7445badbc6a1ac91f"
server_key_version: "encryptionkey@0"
I20260806 02:12:44.061703 23284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:44.079277 23284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:44.082693 23284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:44.082902 23284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:44.083822 23284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:44.084700 23284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:44.084775 23284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:44.084848 23284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:44.084901 23284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:44.093514 23284 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:35727
I20260806 02:12:44.093538 23415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:35727 every 8 connection(s)
I20260806 02:12:44.094089 23284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:12:44.095830 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 23284
I20260806 02:12:44.095990 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:12:44.096288 18713 external_mini_cluster.cc:1428] Setting key 89fa9db3241e93db21c4246dfaa18dab
Aug 06 02:12:44 dist-test-slave-6k22 krb5kdc[22893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982364, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:44.107703 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.095950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:60585 (local address 127.18.70.126:41661)
0806 02:12:44.096091 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:44.096096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:44.097112 (+ 1016us) server_negotiation.cc:407] Connection header received
0806 02:12:44.098613 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:44.098617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:44.098670 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:44.098749 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:44.099962 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.100529 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.101623 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.101847 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.101970 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.104448 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:44.104451 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:44.104453 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:44.104476 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:44.105754 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:44.106236 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:44.106240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:44.106252 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:44.106323 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:44.106658 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:44.106661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:44.106663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:44.106833 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:44.106935 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:44.107393 (+ 458us) server_negotiation.cc:299] Negotiation successful
0806 02:12:44.107506 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 02:12:44.108453 23418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.096408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:41661 (local address 127.18.70.67:60585)
0806 02:12:44.096957 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:44.097019 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:44.098405 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:44.098930 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:44.098937 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:44.099270 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:44.099818 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:44.099833 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.100653 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.100656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:44.101496 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:44.101505 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.102023 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.102028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:44.102200 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.102732 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:44.102757 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:44.104278 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:44.105868 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:44.105873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:44.105887 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:44.106130 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:44.106414 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:44.106418 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:44.106419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:44.106552 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:44.106935 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:44.106943 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:44.107168 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 02:12:44.107350 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 02:12:44.107663 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":160,"threads_started":1}
I20260806 02:12:44.109625 23416 heartbeater.cc:347] Connected to a master server at 127.18.70.126:41661
I20260806 02:12:44.109917 23416 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:44.110419 23416 heartbeater.cc:511] Master 127.18.70.126:41661 requested a full tablet report, sending...
I20260806 02:12:44.111339 22937 ts_manager.cc:194] Registered new tserver with Master: 58495175fa6d45038bbdf906d5ae3ae5 (127.18.70.67:35727)
I20260806 02:12:44.111994 22937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:60585
I20260806 02:12:44.121690 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:12:44.130447 22980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.123711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.18.70.126:41661)
0806 02:12:44.123832 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:44.123836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:44.124008 (+ 172us) server_negotiation.cc:407] Connection header received
0806 02:12:44.124098 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:44.124100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:44.124150 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:44.124215 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:44.124891 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.125368 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.126163 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.126374 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.126444 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.127775 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:44.127777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:44.127779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:44.127796 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:44.129084 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:44.129448 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:44.129450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:44.129451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:44.129517 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:44.129761 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:44.129763 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:44.129764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:44.129942 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:44.130037 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:44.130226 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 02:12:44.130346 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 02:12:44.133106 22937 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:12:44.136454 22937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:44.150475 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.144740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:36285 (local address 127.0.0.1:36120)
0806 02:12:44.145063 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:44.145109 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:44.145229 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:44.145894 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:44.145900 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:44.145925 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:44.146062 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:44.146068 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.147315 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.147320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:44.148803 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:44.148815 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.149891 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.149894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:44.150000 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.150041 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 02:12:44.150159 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 02:12:44.150273 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":18,"thread_start_us":97,"threads_started":1}
I20260806 02:12:44.150502 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.144896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36120 (local address 127.18.70.66:36285)
0806 02:12:44.145383 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:44.145396 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:44.145419 (+ 23us) server_negotiation.cc:407] Connection header received
0806 02:12:44.145502 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:44.145507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:44.145735 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:44.145892 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:44.146183 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.147122 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.148949 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.149758 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.149873 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.150010 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:44.150248 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 02:12:44.150348 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":158,"threads_started":1}
I20260806 02:12:44.150925 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.144954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:35727 (local address 127.0.0.1:33830)
0806 02:12:44.145647 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:44.145660 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:44.145756 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:44.146994 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:44.146998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:44.147015 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:44.147120 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:44.147126 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.148357 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.148363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:44.149466 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:44.149476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.150499 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.150505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:44.150630 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.150665 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 02:12:44.150723 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 02:12:44.150788 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":93,"threads_started":1}
I20260806 02:12:44.151662 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.145696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33830 (local address 127.18.70.67:35727)
0806 02:12:44.146556 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:44.146574 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:44.146596 (+ 22us) server_negotiation.cc:407] Connection header received
0806 02:12:44.146669 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:44.146675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:44.146851 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:44.146997 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:44.147236 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.148134 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.149593 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.150357 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.151118 (+ 761us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.151300 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:44.151391 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 02:12:44.151473 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":151,"threads_started":1}
I20260806 02:12:44.151916 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.145491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:36993 (local address 127.0.0.1:42252)
0806 02:12:44.146418 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:44.146431 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:44.146515 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:44.147347 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:44.147350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:44.147388 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:44.147520 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:44.147527 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.149049 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.149052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:44.150186 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:44.150199 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.151450 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.151454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:44.151549 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.151609 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 02:12:44.151678 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 02:12:44.151774 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"mutex_wait_us":29,"thread_start_us":96,"threads_started":1}
I20260806 02:12:44.152372 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.146067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42252 (local address 127.18.70.65:36993)
0806 02:12:44.146741 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:44.146759 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:44.146777 (+ 18us) server_negotiation.cc:407] Connection header received
0806 02:12:44.146841 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:44.146846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:44.147001 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:44.147160 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:44.147667 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.148913 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.150331 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.151147 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.151245 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.151383 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:44.152125 (+ 742us) server_negotiation.cc:299] Negotiation successful
0806 02:12:44.152216 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":76,"threads_started":1}
I20260806 02:12:44.153868 23211 tablet_service.cc:1467] Processing CreateTablet for tablet 538bcdc9ad374e63b9155aa451d9fd2f (DEFAULT_TABLE table=test-table [id=478c635ce43044ddaa936c1d3dd495b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:44.154124 23348 tablet_service.cc:1467] Processing CreateTablet for tablet 538bcdc9ad374e63b9155aa451d9fd2f (DEFAULT_TABLE table=test-table [id=478c635ce43044ddaa936c1d3dd495b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:44.154958 23348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538bcdc9ad374e63b9155aa451d9fd2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:44.154983 23211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538bcdc9ad374e63b9155aa451d9fd2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:44.154937 23069 tablet_service.cc:1467] Processing CreateTablet for tablet 538bcdc9ad374e63b9155aa451d9fd2f (DEFAULT_TABLE table=test-table [id=478c635ce43044ddaa936c1d3dd495b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:44.155910 23069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538bcdc9ad374e63b9155aa451d9fd2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:44.161267 23433 tablet_bootstrap.cc:492] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5: Bootstrap starting.
I20260806 02:12:44.162676 23434 tablet_bootstrap.cc:492] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65: Bootstrap starting.
I20260806 02:12:44.163617 23433 tablet_bootstrap.cc:654] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:44.163887 23435 tablet_bootstrap.cc:492] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5: Bootstrap starting.
I20260806 02:12:44.164942 23433 log.cc:824] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:44.165583 23434 tablet_bootstrap.cc:654] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:44.165961 23435 tablet_bootstrap.cc:654] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:44.166446 23434 log.cc:824] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:44.167192 23435 log.cc:824] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:44.167287 23433 tablet_bootstrap.cc:492] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5: No bootstrap required, opened a new log
I20260806 02:12:44.167470 23433 ts_tablet_manager.cc:1397] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 02:12:44.168394 23434 tablet_bootstrap.cc:492] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65: No bootstrap required, opened a new log
I20260806 02:12:44.168573 23434 ts_tablet_manager.cc:1397] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 02:12:44.169137 23435 tablet_bootstrap.cc:492] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5: No bootstrap required, opened a new log
I20260806 02:12:44.169317 23435 ts_tablet_manager.cc:1397] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 02:12:44.170692 23433 raft_consensus.cc:348] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afc524267214d8cac7489369a8f2c65" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36285 } } peers { permanent_uuid: "598b9d2666ff453599e083503a20e9d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36993 } } peers { permanent_uuid: "58495175fa6d45038bbdf906d5ae3ae5" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35727 } }
I20260806 02:12:44.170960 23433 raft_consensus.cc:374] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:44.171056 23433 raft_consensus.cc:729] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58495175fa6d45038bbdf906d5ae3ae5, State: Initialized, Role: FOLLOWER
I20260806 02:12:44.171509 23433 consensus_queue.cc:260] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afc524267214d8cac7489369a8f2c65" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36285 } } peers { permanent_uuid: "598b9d2666ff453599e083503a20e9d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36993 } } peers { permanent_uuid: "58495175fa6d45038bbdf906d5ae3ae5" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35727 } }
I20260806 02:12:44.172209 23416 heartbeater.cc:503] Master 127.18.70.126:41661 was elected leader, sending a full tablet report...
I20260806 02:12:44.172541 23433 ts_tablet_manager.cc:1428] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 02:12:44.173045 23435 raft_consensus.cc:348] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afc524267214d8cac7489369a8f2c65" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36285 } } peers { permanent_uuid: "598b9d2666ff453599e083503a20e9d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36993 } } peers { permanent_uuid: "58495175fa6d45038bbdf906d5ae3ae5" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35727 } }
I20260806 02:12:44.173354 23435 raft_consensus.cc:374] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:44.173430 23435 raft_consensus.cc:729] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 598b9d2666ff453599e083503a20e9d5, State: Initialized, Role: FOLLOWER
I20260806 02:12:44.173861 23435 consensus_queue.cc:260] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afc524267214d8cac7489369a8f2c65" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36285 } } peers { permanent_uuid: "598b9d2666ff453599e083503a20e9d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36993 } } peers { permanent_uuid: "58495175fa6d45038bbdf906d5ae3ae5" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35727 } }
I20260806 02:12:44.174506 23139 heartbeater.cc:503] Master 127.18.70.126:41661 was elected leader, sending a full tablet report...
I20260806 02:12:44.174844 23435 ts_tablet_manager.cc:1428] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 02:12:44.172384 23434 raft_consensus.cc:348] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afc524267214d8cac7489369a8f2c65" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36285 } } peers { permanent_uuid: "598b9d2666ff453599e083503a20e9d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36993 } } peers { permanent_uuid: "58495175fa6d45038bbdf906d5ae3ae5" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35727 } }
I20260806 02:12:44.177166 23434 raft_consensus.cc:374] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:44.177259 23434 raft_consensus.cc:729] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5afc524267214d8cac7489369a8f2c65, State: Initialized, Role: FOLLOWER
I20260806 02:12:44.177662 23434 consensus_queue.cc:260] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afc524267214d8cac7489369a8f2c65" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36285 } } peers { permanent_uuid: "598b9d2666ff453599e083503a20e9d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36993 } } peers { permanent_uuid: "58495175fa6d45038bbdf906d5ae3ae5" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35727 } }
I20260806 02:12:44.178447 23278 heartbeater.cc:503] Master 127.18.70.126:41661 was elected leader, sending a full tablet report...
I20260806 02:12:44.178663 23434 ts_tablet_manager.cc:1428] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
W20260806 02:12:44.345002 23417 tablet.cc:2369] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:12:44.361756 23441 raft_consensus.cc:482] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:44.362018 23441 raft_consensus.cc:504] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afc524267214d8cac7489369a8f2c65" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36285 } } peers { permanent_uuid: "598b9d2666ff453599e083503a20e9d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36993 } } peers { permanent_uuid: "58495175fa6d45038bbdf906d5ae3ae5" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35727 } }
I20260806 02:12:44.363304 23441 leader_election.cc:288] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5afc524267214d8cac7489369a8f2c65 (127.18.70.66:36285), 58495175fa6d45038bbdf906d5ae3ae5 (127.18.70.67:35727)
I20260806 02:12:44.367623 23444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.363544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:36285 (local address 127.18.70.65:57761)
0806 02:12:44.363955 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:44.363967 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:44.364068 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:44.364458 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:44.364461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:44.364478 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:44.364644 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:44.364651 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.365523 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.365527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:44.366471 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:44.366481 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.367157 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.367163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:44.367255 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.367296 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 02:12:44.367365 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 02:12:44.367421 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":3,"thread_start_us":145,"threads_started":1}
I20260806 02:12:44.367636 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.363656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:57761 (local address 127.18.70.66:36285)
0806 02:12:44.363800 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:44.363804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:44.364053 (+ 249us) server_negotiation.cc:407] Connection header received
0806 02:12:44.364182 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:44.364185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:44.364240 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:44.364322 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:44.364781 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.365405 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.366595 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.367035 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.367129 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.367209 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:44.367402 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 02:12:44.367488 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:12:44.367709 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.363590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:35727 (local address 127.18.70.65:54211)
0806 02:12:44.364076 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:44.364091 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:44.364214 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:44.364636 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:44.364639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:44.364663 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:44.364793 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:44.364801 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.365603 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.365605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:44.366506 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:44.366523 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.367219 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.367226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:44.367336 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.367381 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 02:12:44.367459 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 02:12:44.367521 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":114,"threads_started":1}
I20260806 02:12:44.367718 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.363722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:54211 (local address 127.18.70.67:35727)
0806 02:12:44.363882 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:44.363888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:44.364221 (+ 333us) server_negotiation.cc:407] Connection header received
0806 02:12:44.364456 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:44.364459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:44.364511 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:44.364592 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:44.364957 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.365477 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.366664 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.367067 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.367203 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.367257 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:44.367499 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 02:12:44.367593 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 02:12:44.368294 23231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538bcdc9ad374e63b9155aa451d9fd2f" candidate_uuid: "598b9d2666ff453599e083503a20e9d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5afc524267214d8cac7489369a8f2c65" is_pre_election: true
I20260806 02:12:44.368353 23368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538bcdc9ad374e63b9155aa451d9fd2f" candidate_uuid: "598b9d2666ff453599e083503a20e9d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58495175fa6d45038bbdf906d5ae3ae5" is_pre_election: true
I20260806 02:12:44.368611 23231 raft_consensus.cc:2454] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 598b9d2666ff453599e083503a20e9d5 in term 0.
I20260806 02:12:44.368662 23368 raft_consensus.cc:2454] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 598b9d2666ff453599e083503a20e9d5 in term 0.
I20260806 02:12:44.369140 23019 leader_election.cc:302] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 598b9d2666ff453599e083503a20e9d5, 5afc524267214d8cac7489369a8f2c65; no voters:
I20260806 02:12:44.369441 23441 raft_consensus.cc:2781] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:44.369530 23441 raft_consensus.cc:482] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:44.369572 23441 raft_consensus.cc:3037] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:44.370594 23441 raft_consensus.cc:504] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afc524267214d8cac7489369a8f2c65" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36285 } } peers { permanent_uuid: "598b9d2666ff453599e083503a20e9d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36993 } } peers { permanent_uuid: "58495175fa6d45038bbdf906d5ae3ae5" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35727 } }
I20260806 02:12:44.370942 23441 leader_election.cc:288] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [CANDIDATE]: Term 1 election: Requested vote from peers 5afc524267214d8cac7489369a8f2c65 (127.18.70.66:36285), 58495175fa6d45038bbdf906d5ae3ae5 (127.18.70.67:35727)
I20260806 02:12:44.371258 23231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538bcdc9ad374e63b9155aa451d9fd2f" candidate_uuid: "598b9d2666ff453599e083503a20e9d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5afc524267214d8cac7489369a8f2c65"
I20260806 02:12:44.371325 23368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538bcdc9ad374e63b9155aa451d9fd2f" candidate_uuid: "598b9d2666ff453599e083503a20e9d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58495175fa6d45038bbdf906d5ae3ae5"
I20260806 02:12:44.371384 23231 raft_consensus.cc:3037] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:44.371475 23368 raft_consensus.cc:3037] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:44.372421 23231 raft_consensus.cc:2454] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 598b9d2666ff453599e083503a20e9d5 in term 1.
I20260806 02:12:44.372520 23368 raft_consensus.cc:2454] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 598b9d2666ff453599e083503a20e9d5 in term 1.
I20260806 02:12:44.372783 23019 leader_election.cc:302] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 598b9d2666ff453599e083503a20e9d5, 5afc524267214d8cac7489369a8f2c65; no voters:
I20260806 02:12:44.372934 23441 raft_consensus.cc:2781] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [term 1 FOLLOWER]: Leader election won for term 1
W20260806 02:12:44.373055 23279 tablet.cc:2369] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:12:44.373214 23441 raft_consensus.cc:686] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [term 1 LEADER]: Becoming Leader. State: Replica: 598b9d2666ff453599e083503a20e9d5, State: Running, Role: LEADER
I20260806 02:12:44.373571 23441 consensus_queue.cc:237] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afc524267214d8cac7489369a8f2c65" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36285 } } peers { permanent_uuid: "598b9d2666ff453599e083503a20e9d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36993 } } peers { permanent_uuid: "58495175fa6d45038bbdf906d5ae3ae5" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35727 } }
I20260806 02:12:44.376762 22938 catalog_manager.cc:5495] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 598b9d2666ff453599e083503a20e9d5 (127.18.70.65). New cstate: current_term: 1 leader_uuid: "598b9d2666ff453599e083503a20e9d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5afc524267214d8cac7489369a8f2c65" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "598b9d2666ff453599e083503a20e9d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58495175fa6d45038bbdf906d5ae3ae5" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35727 } health_report { overall_health: UNKNOWN } } }
W20260806 02:12:44.406579 23140 tablet.cc:2369] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:12:44.440085 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.436815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42254 (local address 127.18.70.65:36993)
0806 02:12:44.436951 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:44.436955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:44.436967 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:12:44.437042 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:44.437044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:44.437113 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:44.437203 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:44.437425 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.437907 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.438714 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:44.438920 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:44.438994 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:44.439266 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:44.439728 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:44.439839 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:44.439930 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 02:12:44.439978 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 02:12:44.445640 23368 raft_consensus.cc:1261] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5 [term 1 FOLLOWER]: Refusing update from remote peer 598b9d2666ff453599e083503a20e9d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:44.445641 23231 raft_consensus.cc:1261] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65 [term 1 FOLLOWER]: Refusing update from remote peer 598b9d2666ff453599e083503a20e9d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:44.446378 23445 consensus_queue.cc:1035] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58495175fa6d45038bbdf906d5ae3ae5" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:44.446545 23441 consensus_queue.cc:1035] T 538bcdc9ad374e63b9155aa451d9fd2f P 598b9d2666ff453599e083503a20e9d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5afc524267214d8cac7489369a8f2c65" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:44.454588 23450 mvcc.cc:205] Tried to move back new op lower bound from 7315383764762005504 to 7315383764476788736. Current Snapshot: MvccSnapshot[applied={T|T < 7315383764762005504}]
I20260806 02:12:44.455516 23449 mvcc.cc:205] Tried to move back new op lower bound from 7315383764762005504 to 7315383764476788736. Current Snapshot: MvccSnapshot[applied={T|T < 7315383764762005504}]
I20260806 02:12:44.462704 22936 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:44.462838 22936 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:44.465250 22936 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9626bca2305d4912b5544aae0d1e5169: Sending DeleteTablet for 3 replicas of tablet 538bcdc9ad374e63b9155aa451d9fd2f
I20260806 02:12:44.466089 23348 tablet_service.cc:1514] Processing DeleteTablet for tablet 538bcdc9ad374e63b9155aa451d9fd2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:44 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:33830
I20260806 02:12:44.466532 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 23003
I20260806 02:12:44.466610 23211 tablet_service.cc:1514] Processing DeleteTablet for tablet 538bcdc9ad374e63b9155aa451d9fd2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:44 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:36120
I20260806 02:12:44.467595 23458 tablet_replica.cc:330] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5: stopping tablet replica
I20260806 02:12:44.467893 23459 tablet_replica.cc:330] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65: stopping tablet replica
I20260806 02:12:44.468335 23459 raft_consensus.cc:2229] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:44.468683 23458 raft_consensus.cc:2229] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:44.469003 23458 raft_consensus.cc:2258] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:44.469323 23459 raft_consensus.cc:2258] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:44.471045 23458 ts_tablet_manager.cc:1905] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:44.471092 23459 ts_tablet_manager.cc:1905] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:44.473325 23458 ts_tablet_manager.cc:1918] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:44.473435 23458 log.cc:1196] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/538bcdc9ad374e63b9155aa451d9fd2f
I20260806 02:12:44.473726 23458 ts_tablet_manager.cc:1939] T 538bcdc9ad374e63b9155aa451d9fd2f P 58495175fa6d45038bbdf906d5ae3ae5: Deleting consensus metadata
I20260806 02:12:44.474419 23459 ts_tablet_manager.cc:1918] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:44.474516 23459 log.cc:1196] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/538bcdc9ad374e63b9155aa451d9fd2f
I20260806 02:12:44.474815 23459 ts_tablet_manager.cc:1939] T 538bcdc9ad374e63b9155aa451d9fd2f P 5afc524267214d8cac7489369a8f2c65: Deleting consensus metadata
I20260806 02:12:44.474967 22925 catalog_manager.cc:4841] TS 58495175fa6d45038bbdf906d5ae3ae5 (127.18.70.67:35727): tablet 538bcdc9ad374e63b9155aa451d9fd2f (table test-table [id=478c635ce43044ddaa936c1d3dd495b1]) successfully deleted
I20260806 02:12:44.476013 22923 catalog_manager.cc:4841] TS 5afc524267214d8cac7489369a8f2c65 (127.18.70.66:36285): tablet 538bcdc9ad374e63b9155aa451d9fd2f (table test-table [id=478c635ce43044ddaa936c1d3dd495b1]) successfully deleted
W20260806 02:12:44.477820 22923 connection.cc:535] client connection to 127.18.70.65:36993 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:12:44.477988 22923 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 02:12:44.478034 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 23144
I20260806 02:12:44.480044 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.479711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:36993 (local address 127.0.0.1:42260)
0806 02:12:44.479861 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:44.479943 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:36993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 02:12:44.480206 22923 catalog_manager.cc:4584] TS 598b9d2666ff453599e083503a20e9d5 (127.18.70.65:36993): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 538bcdc9ad374e63b9155aa451d9fd2f: Network error: Client connection negotiation failed: client connection to 127.18.70.65:36993: connect: Connection refused (error 111)
I20260806 02:12:44.485980 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 23284
I20260806 02:12:44.492584 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 22907
2026-08-06T02:12:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2527 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T02:12:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:44Z Disabled control of system clock
I20260806 02:12:44.524411 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:37493
--webserver_interface=127.18.70.126
--webserver_port=0
--builtin_ntp_servers=127.18.70.84:37709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:37493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 02:12:44.654769 23464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:44.655006 23464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:44.655041 23464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:44.658942 23464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:44.658989 23464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:44.659004 23464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:44.659021 23464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:44.659037 23464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:44.663552 23464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:37493
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:37493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:44.664613 23464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:44.666040 23464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:44.674801 23469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:44.674801 23472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:44.674857 23470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:44.675194 23464 server_base.cc:1034] running on GCE node
I20260806 02:12:44.675840 23464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:44.676867 23464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:44.678061 23464 hybrid_clock.cc:648] HybridClock initialized: now 1785982364678041 us; error 30 us; skew 500 ppm
I20260806 02:12:44.680006 23464 webserver.cc:460] Webserver started at http://127.18.70.126:38209/ using document root <none> and password file <none>
I20260806 02:12:44.680594 23464 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:44.680683 23464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:44.680914 23464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:44.682895 23464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "c01ec59e92de4de883ce2389f2965938"
format_stamp: "Formatted at 2026-08-06 02:12:44 on dist-test-slave-6k22"
server_key: "6dbebd9b0c2b8b2fdb4bf0d91629d7b6"
server_key_iv: "3fec62d49f46e2c060ae2a39564be0ec"
server_key_version: "encryptionkey@0"
I20260806 02:12:44.683434 23464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "c01ec59e92de4de883ce2389f2965938"
format_stamp: "Formatted at 2026-08-06 02:12:44 on dist-test-slave-6k22"
server_key: "6dbebd9b0c2b8b2fdb4bf0d91629d7b6"
server_key_iv: "3fec62d49f46e2c060ae2a39564be0ec"
server_key_version: "encryptionkey@0"
I20260806 02:12:44.687006 23464 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 02:12:44.689280 23478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:44.690483 23464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:44.690622 23464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "c01ec59e92de4de883ce2389f2965938"
format_stamp: "Formatted at 2026-08-06 02:12:44 on dist-test-slave-6k22"
server_key: "6dbebd9b0c2b8b2fdb4bf0d91629d7b6"
server_key_iv: "3fec62d49f46e2c060ae2a39564be0ec"
server_key_version: "encryptionkey@0"
I20260806 02:12:44.690713 23464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:44.695151 23464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:44.695787 23464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:44.695959 23464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:44.702903 23464 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:37493
I20260806 02:12:44.702915 23532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:37493 every 8 connection(s)
I20260806 02:12:44.703871 23464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:44.707063 23533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:44.709936 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 23464
I20260806 02:12:44.710027 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:44.710310 18713 external_mini_cluster.cc:1428] Setting key 479497b12601a105f161daf33c03fd9c
I20260806 02:12:44.712931 23533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938: Bootstrap starting.
I20260806 02:12:44.715332 23533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:44.715946 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.711776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55372 (local address 127.18.70.126:37493)
0806 02:12:44.712405 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:44.712423 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:44.712457 (+ 34us) server_negotiation.cc:407] Connection header received
0806 02:12:44.713060 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:44.713085 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:44.713421 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:44.713791 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:44.714546 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:44.714551 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:44.714562 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:44.714573 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:44.714625 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:44.714640 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:44.714763 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:44.714866 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 02:12:44.715105 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":141,"threads_started":1}
I20260806 02:12:44.716279 23533 log.cc:824] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:44.718266 23533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938: No bootstrap required, opened a new log
I20260806 02:12:44.721907 23533 raft_consensus.cc:348] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01ec59e92de4de883ce2389f2965938" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37493 } }
I20260806 02:12:44.722146 23533 raft_consensus.cc:374] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:44.722221 23533 raft_consensus.cc:729] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01ec59e92de4de883ce2389f2965938, State: Initialized, Role: FOLLOWER
I20260806 02:12:44.722738 23533 consensus_queue.cc:260] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01ec59e92de4de883ce2389f2965938" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37493 } }
I20260806 02:12:44.722883 23533 raft_consensus.cc:388] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:44.722930 23533 raft_consensus.cc:482] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:44.723001 23533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:44.723855 23533 raft_consensus.cc:504] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01ec59e92de4de883ce2389f2965938" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37493 } }
I20260806 02:12:44.724140 23533 leader_election.cc:302] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c01ec59e92de4de883ce2389f2965938; no voters:
I20260806 02:12:44.724387 23533 leader_election.cc:288] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:44.724538 23538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:44.724768 23538 raft_consensus.cc:686] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [term 1 LEADER]: Becoming Leader. State: Replica: c01ec59e92de4de883ce2389f2965938, State: Running, Role: LEADER
I20260806 02:12:44.725063 23538 consensus_queue.cc:237] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01ec59e92de4de883ce2389f2965938" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37493 } }
I20260806 02:12:44.725597 23533 sys_catalog.cc:564] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:44.726888 23539 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c01ec59e92de4de883ce2389f2965938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01ec59e92de4de883ce2389f2965938" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37493 } } }
I20260806 02:12:44.726889 23540 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c01ec59e92de4de883ce2389f2965938. Latest consensus state: current_term: 1 leader_uuid: "c01ec59e92de4de883ce2389f2965938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01ec59e92de4de883ce2389f2965938" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37493 } } }
I20260806 02:12:44.727051 23539 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:44.727073 23540 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:44.727502 23547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:44.730742 23547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:44.736341 23547 catalog_manager.cc:1347] Generated new cluster ID: 7f7046f69b5640c2adb3fd1abcfcfa21
I20260806 02:12:44.736411 23547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:44.747907 23547 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:44.748216 23547 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:44.756819 23547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938: Generated new TSK 0
I20260806 02:12:44.757395 23547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:12:44.762223 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37493
--builtin_ntp_servers=127.18.70.84:37709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 02:12:44.873801 23557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:44.874025 23557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:44.874063 23557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:44.878269 23557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:44.878315 23557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:44.878412 23557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:12:44.883067 23557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:44.884106 23557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:44.885440 23557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:44.894660 23563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:44.894876 23565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:44.894671 23562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:44.894920 23557 server_base.cc:1034] running on GCE node
I20260806 02:12:44.895419 23557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:44.896101 23557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:44.897261 23557 hybrid_clock.cc:648] HybridClock initialized: now 1785982364897233 us; error 39 us; skew 500 ppm
I20260806 02:12:44.899085 23557 webserver.cc:460] Webserver started at http://127.18.70.65:36701/ using document root <none> and password file <none>
I20260806 02:12:44.899698 23557 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:44.899765 23557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:44.900002 23557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:44.901965 23557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "cc04c752fbdb464cb95353b57c5a48d5"
format_stamp: "Formatted at 2026-08-06 02:12:44 on dist-test-slave-6k22"
server_key: "a04fe741029fdc59cfded7a5f1f8acc3"
server_key_iv: "c9e4b173c88a861af6e79347801d2d41"
server_key_version: "encryptionkey@0"
I20260806 02:12:44.902499 23557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "cc04c752fbdb464cb95353b57c5a48d5"
format_stamp: "Formatted at 2026-08-06 02:12:44 on dist-test-slave-6k22"
server_key: "a04fe741029fdc59cfded7a5f1f8acc3"
server_key_iv: "c9e4b173c88a861af6e79347801d2d41"
server_key_version: "encryptionkey@0"
I20260806 02:12:44.906090 23557 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 02:12:44.908588 23571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:44.909677 23557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:44.909816 23557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "cc04c752fbdb464cb95353b57c5a48d5"
format_stamp: "Formatted at 2026-08-06 02:12:44 on dist-test-slave-6k22"
server_key: "a04fe741029fdc59cfded7a5f1f8acc3"
server_key_iv: "c9e4b173c88a861af6e79347801d2d41"
server_key_version: "encryptionkey@0"
I20260806 02:12:44.909924 23557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:44.914266 23557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:44.915069 23557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:44.915277 23557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:44.915908 23557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:44.916801 23557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:44.916872 23557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:44.916952 23557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:44.916996 23557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:44.926752 23557 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:33065
I20260806 02:12:44.926795 23689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:33065 every 8 connection(s)
I20260806 02:12:44.927726 23557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:12:44.928969 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 23557
I20260806 02:12:44.929112 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:12:44.929383 18713 external_mini_cluster.cc:1428] Setting key 8a65cd6b28b5f673e5f4fd8fdbd286e9
I20260806 02:12:44.931640 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37493
--builtin_ntp_servers=127.18.70.84:37709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 02:12:44.934486 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.929530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:53857 (local address 127.18.70.126:37493)
0806 02:12:44.929697 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:44.929702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:44.930824 (+ 1122us) server_negotiation.cc:407] Connection header received
0806 02:12:44.932754 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:44.932757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:44.932790 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:44.932856 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:44.933988 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:44.933992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:44.933994 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:44.934006 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:44.934027 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:44.934035 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:44.934128 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:44.934314 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 02:12:44.934358 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 02:12:44.935310 23692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:44.929993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:37493 (local address 127.18.70.65:53857)
0806 02:12:44.930660 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:44.930741 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:44.932537 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:44.933014 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:44.933024 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:44.933140 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:44.933750 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:12:44.933752 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:44.933788 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:12:44.933851 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:44.934139 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:44.934150 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:44.934156 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 02:12:44.934357 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 02:12:44.934591 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":140,"threads_started":1}
I20260806 02:12:44.936685 23690 heartbeater.cc:347] Connected to a master server at 127.18.70.126:37493
I20260806 02:12:44.936909 23690 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:44.937320 23690 heartbeater.cc:511] Master 127.18.70.126:37493 requested a full tablet report, sending...
I20260806 02:12:44.939072 23495 ts_manager.cc:194] Registered new tserver with Master: cc04c752fbdb464cb95353b57c5a48d5 (127.18.70.65:33065)
W20260806 02:12:45.045069 23693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:45.045307 23693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:45.045346 23693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:45.049173 23693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:45.049237 23693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:45.049294 23693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:12:45.053722 23693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:45.054702 23693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:45.056118 23693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:45.065919 23698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:45.065876 23699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:45.065886 23701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:45.066191 23693 server_base.cc:1034] running on GCE node
I20260806 02:12:45.066679 23693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:45.067416 23693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:45.068608 23693 hybrid_clock.cc:648] HybridClock initialized: now 1785982365068587 us; error 31 us; skew 500 ppm
I20260806 02:12:45.070501 23693 webserver.cc:460] Webserver started at http://127.18.70.66:35191/ using document root <none> and password file <none>
I20260806 02:12:45.071193 23693 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:45.071296 23693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:45.071591 23693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:45.073602 23693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "cfc89e8a34d34865b9f7956379d54d94"
format_stamp: "Formatted at 2026-08-06 02:12:45 on dist-test-slave-6k22"
server_key: "f5dd86c1490c874e1b339f72b00ee601"
server_key_iv: "c7216dcbe2db38720bbe3030db2745ab"
server_key_version: "encryptionkey@0"
I20260806 02:12:45.074167 23693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "cfc89e8a34d34865b9f7956379d54d94"
format_stamp: "Formatted at 2026-08-06 02:12:45 on dist-test-slave-6k22"
server_key: "f5dd86c1490c874e1b339f72b00ee601"
server_key_iv: "c7216dcbe2db38720bbe3030db2745ab"
server_key_version: "encryptionkey@0"
I20260806 02:12:45.078078 23693 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 02:12:45.080613 23707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:45.081786 23693 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:45.081929 23693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "cfc89e8a34d34865b9f7956379d54d94"
format_stamp: "Formatted at 2026-08-06 02:12:45 on dist-test-slave-6k22"
server_key: "f5dd86c1490c874e1b339f72b00ee601"
server_key_iv: "c7216dcbe2db38720bbe3030db2745ab"
server_key_version: "encryptionkey@0"
I20260806 02:12:45.082044 23693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:45.086644 23693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:45.087267 23693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:45.087443 23693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:45.088014 23693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:45.088984 23693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:45.089051 23693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:45.089119 23693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:45.089170 23693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:45.098497 23693 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:38251
I20260806 02:12:45.098518 23828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:38251 every 8 connection(s)
I20260806 02:12:45.099465 23693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:12:45.105595 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.101344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:54159 (local address 127.18.70.126:37493)
0806 02:12:45.101511 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:45.101515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:45.102565 (+ 1050us) server_negotiation.cc:407] Connection header received
0806 02:12:45.103998 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:45.104001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:45.104040 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:45.104115 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:45.105229 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:45.105231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:45.105233 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:45.105241 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:45.105256 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:45.105263 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:45.105340 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:45.105442 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 02:12:45.105481 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 02:12:45.106456 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.101761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:37493 (local address 127.18.70.66:54159)
0806 02:12:45.102420 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:45.102460 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:45.103802 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:45.104274 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:45.104281 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:45.104369 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:45.104990 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:12:45.104992 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:45.105033 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:12:45.105101 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:45.105331 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:45.105335 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:45.105339 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 02:12:45.105460 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 02:12:45.105678 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":115,"threads_started":1}
I20260806 02:12:45.107805 23830 heartbeater.cc:347] Connected to a master server at 127.18.70.126:37493
I20260806 02:12:45.108044 23830 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:45.108462 23830 heartbeater.cc:511] Master 127.18.70.126:37493 requested a full tablet report, sending...
I20260806 02:12:45.109165 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 23693
I20260806 02:12:45.109330 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:12:45.109685 18713 external_mini_cluster.cc:1428] Setting key dff7aceb6326ad643119b5589a24cc2b
I20260806 02:12:45.109675 23495 ts_manager.cc:194] Registered new tserver with Master: cfc89e8a34d34865b9f7956379d54d94 (127.18.70.66:38251)
I20260806 02:12:45.111629 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37493
--builtin_ntp_servers=127.18.70.84:37709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 02:12:45.224427 23833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:45.224651 23833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:45.224689 23833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:45.228845 23833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:45.228891 23833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:45.228941 23833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:12:45.233208 23833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:45.234069 23833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:45.235422 23833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:45.244964 23842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:45.244951 23840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:45.245005 23839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:45.245141 23833 server_base.cc:1034] running on GCE node
I20260806 02:12:45.245631 23833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:45.246219 23833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:45.247378 23833 hybrid_clock.cc:648] HybridClock initialized: now 1785982365247364 us; error 25 us; skew 500 ppm
I20260806 02:12:45.249085 23833 webserver.cc:460] Webserver started at http://127.18.70.67:33341/ using document root <none> and password file <none>
I20260806 02:12:45.249630 23833 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:45.249689 23833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:45.249934 23833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:45.251926 23833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b"
format_stamp: "Formatted at 2026-08-06 02:12:45 on dist-test-slave-6k22"
server_key: "f71c476e891173569a7214bd79ab8fd4"
server_key_iv: "9875cd93b839b982f27cb2374752078e"
server_key_version: "encryptionkey@0"
I20260806 02:12:45.252449 23833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b"
format_stamp: "Formatted at 2026-08-06 02:12:45 on dist-test-slave-6k22"
server_key: "f71c476e891173569a7214bd79ab8fd4"
server_key_iv: "9875cd93b839b982f27cb2374752078e"
server_key_version: "encryptionkey@0"
I20260806 02:12:45.256052 23833 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:12:45.258461 23849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:45.259709 23833 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:45.259835 23833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b"
format_stamp: "Formatted at 2026-08-06 02:12:45 on dist-test-slave-6k22"
server_key: "f71c476e891173569a7214bd79ab8fd4"
server_key_iv: "9875cd93b839b982f27cb2374752078e"
server_key_version: "encryptionkey@0"
I20260806 02:12:45.259933 23833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:45.264703 23833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:45.265280 23833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:45.265470 23833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:45.265990 23833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:45.266906 23833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:45.266950 23833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:45.267014 23833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:45.267050 23833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:45.275682 23833 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:36189
I20260806 02:12:45.275731 23970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:36189 every 8 connection(s)
I20260806 02:12:45.276547 23833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:12:45.278198 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 23833
I20260806 02:12:45.278328 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:12:45.278625 18713 external_mini_cluster.cc:1428] Setting key dd366d44a33b597cb0583e975381a5fe
I20260806 02:12:45.282790 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.278447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:55981 (local address 127.18.70.126:37493)
0806 02:12:45.278580 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:45.278584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:45.279846 (+ 1262us) server_negotiation.cc:407] Connection header received
0806 02:12:45.281288 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:45.281291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:45.281327 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:45.281394 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:45.282350 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:45.282353 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:45.282355 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:45.282366 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:45.282399 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:45.282407 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:45.282473 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:45.282597 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 02:12:45.282651 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 02:12:45.283650 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.278929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:37493 (local address 127.18.70.67:55981)
0806 02:12:45.279688 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:45.279737 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:45.281103 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:45.281515 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:45.281522 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:45.281604 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:45.282104 (+ 500us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:12:45.282106 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:45.282143 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:12:45.282207 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:45.282479 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:45.282486 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:45.282490 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 02:12:45.282615 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 02:12:45.282860 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":139,"threads_started":1}
I20260806 02:12:45.284596 23971 heartbeater.cc:347] Connected to a master server at 127.18.70.126:37493
I20260806 02:12:45.284821 23971 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:45.285202 23971 heartbeater.cc:511] Master 127.18.70.126:37493 requested a full tablet report, sending...
I20260806 02:12:45.286171 23495 ts_manager.cc:194] Registered new tserver with Master: 8df8c6a5546d4cc599dccc5ba2c21e9b (127.18.70.67:36189)
I20260806 02:12:45.292443 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:12:45.295993 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.294356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55384 (local address 127.18.70.126:37493)
0806 02:12:45.294493 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:45.294498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:45.294638 (+ 140us) server_negotiation.cc:407] Connection header received
0806 02:12:45.294784 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:45.294787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:45.294822 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:45.294911 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:45.295543 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:45.295546 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:45.295549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:45.295579 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:45.295599 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:45.295608 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:45.295719 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:45.295832 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 02:12:45.295872 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 02:12:45.298501 23495 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:12:45.301548 23495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:45.313335 23987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.311139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.18.70.67:36189)
0806 02:12:45.311579 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:45.311591 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:45.311612 (+ 21us) server_negotiation.cc:407] Connection header received
0806 02:12:45.311657 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:45.311662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:45.311836 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:45.311972 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:45.312623 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:45.312628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:45.312635 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:45.312645 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:45.312675 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:45.312717 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:45.312810 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:45.313098 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 02:12:45.313165 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":105,"threads_started":1}
I20260806 02:12:45.313515 23983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.309897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35276 (local address 127.18.70.65:33065)
0806 02:12:45.310258 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:45.310271 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:45.310300 (+ 29us) server_negotiation.cc:407] Connection header received
0806 02:12:45.310489 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:45.310496 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:45.310690 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:45.310832 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:45.312792 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:45.312802 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:45.312812 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:45.312823 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:45.312857 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:45.312878 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:45.312993 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:45.313262 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 02:12:45.313336 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":130,"threads_started":1}
I20260806 02:12:45.313557 23985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.310617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35070 (local address 127.18.70.66:38251)
0806 02:12:45.311097 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:45.311110 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:45.311134 (+ 24us) server_negotiation.cc:407] Connection header received
0806 02:12:45.311253 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:45.311260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:45.311434 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:45.311600 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:45.312544 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:45.312552 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:45.312577 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:45.312591 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:45.312634 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:45.312671 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:45.312810 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:45.313326 (+ 516us) server_negotiation.cc:299] Negotiation successful
0806 02:12:45.313397 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":161,"threads_started":1}
I20260806 02:12:45.313194 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.310429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:38251 (local address 127.0.0.1:35070)
0806 02:12:45.311035 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:45.311048 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:45.311120 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:45.311591 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:45.311598 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:45.311674 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:45.312296 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:12:45.312300 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:45.312322 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:12:45.312387 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:45.312856 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:45.312867 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:45.312873 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 02:12:45.312983 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 02:12:45.313063 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":92,"threads_started":1}
I20260806 02:12:45.313899 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.309703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:33065 (local address 127.0.0.1:35276)
0806 02:12:45.310026 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:45.310090 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:45.310339 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:45.311599 (+ 1260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:45.311602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:45.311673 (+ 71us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:45.312616 (+ 943us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:12:45.312618 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:45.312627 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:12:45.312646 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:45.313039 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:45.313042 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:45.313045 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 02:12:45.313123 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 02:12:45.313159 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
I20260806 02:12:45.314114 23986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.310741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:36189 (local address 127.0.0.1:46164)
0806 02:12:45.311288 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:45.311302 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:45.311376 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:45.311951 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:45.311954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:45.311967 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:45.312430 (+ 463us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:12:45.312436 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:45.312450 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:12:45.312480 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:45.312856 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:45.312867 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:45.312873 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 02:12:45.312983 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 02:12:45.313608 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":540,"thread_start_us":95,"threads_started":1}
I20260806 02:12:45.316648 23621 tablet_service.cc:1467] Processing CreateTablet for tablet 66677b4942bc4e619f727a5656f2185e (DEFAULT_TABLE table=test-table [id=42516aa6cfb34f9ea49700def1219a48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:45.316736 23758 tablet_service.cc:1467] Processing CreateTablet for tablet 66677b4942bc4e619f727a5656f2185e (DEFAULT_TABLE table=test-table [id=42516aa6cfb34f9ea49700def1219a48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:45.316725 23900 tablet_service.cc:1467] Processing CreateTablet for tablet 66677b4942bc4e619f727a5656f2185e (DEFAULT_TABLE table=test-table [id=42516aa6cfb34f9ea49700def1219a48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:45.317613 23621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66677b4942bc4e619f727a5656f2185e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:45.317713 23758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66677b4942bc4e619f727a5656f2185e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:45.317713 23900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66677b4942bc4e619f727a5656f2185e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:45.324085 23988 tablet_bootstrap.cc:492] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5: Bootstrap starting.
I20260806 02:12:45.326138 23990 tablet_bootstrap.cc:492] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94: Bootstrap starting.
I20260806 02:12:45.326725 23988 tablet_bootstrap.cc:654] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:45.325747 23989 tablet_bootstrap.cc:492] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b: Bootstrap starting.
I20260806 02:12:45.327659 23988 log.cc:824] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:45.328527 23990 tablet_bootstrap.cc:654] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:45.329375 23990 log.cc:824] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:45.329873 23989 tablet_bootstrap.cc:654] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:45.330056 23988 tablet_bootstrap.cc:492] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5: No bootstrap required, opened a new log
I20260806 02:12:45.330191 23988 ts_tablet_manager.cc:1397] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:12:45.330629 23989 log.cc:824] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:45.332561 23989 tablet_bootstrap.cc:492] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b: No bootstrap required, opened a new log
I20260806 02:12:45.332762 23989 ts_tablet_manager.cc:1397] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 02:12:45.333187 23990 tablet_bootstrap.cc:492] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94: No bootstrap required, opened a new log
I20260806 02:12:45.333364 23990 ts_tablet_manager.cc:1397] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 02:12:45.334362 23988 raft_consensus.cc:348] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc04c752fbdb464cb95353b57c5a48d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 33065 } } peers { permanent_uuid: "cfc89e8a34d34865b9f7956379d54d94" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38251 } } peers { permanent_uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36189 } }
I20260806 02:12:45.334611 23988 raft_consensus.cc:374] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:45.334666 23988 raft_consensus.cc:729] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc04c752fbdb464cb95353b57c5a48d5, State: Initialized, Role: FOLLOWER
I20260806 02:12:45.335156 23988 consensus_queue.cc:260] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc04c752fbdb464cb95353b57c5a48d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 33065 } } peers { permanent_uuid: "cfc89e8a34d34865b9f7956379d54d94" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38251 } } peers { permanent_uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36189 } }
I20260806 02:12:45.335794 23690 heartbeater.cc:503] Master 127.18.70.126:37493 was elected leader, sending a full tablet report...
I20260806 02:12:45.337074 23988 ts_tablet_manager.cc:1428] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260806 02:12:45.337226 23989 raft_consensus.cc:348] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc04c752fbdb464cb95353b57c5a48d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 33065 } } peers { permanent_uuid: "cfc89e8a34d34865b9f7956379d54d94" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38251 } } peers { permanent_uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36189 } }
I20260806 02:12:45.337476 23989 raft_consensus.cc:374] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:45.337533 23989 raft_consensus.cc:729] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df8c6a5546d4cc599dccc5ba2c21e9b, State: Initialized, Role: FOLLOWER
I20260806 02:12:45.337715 23990 raft_consensus.cc:348] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc04c752fbdb464cb95353b57c5a48d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 33065 } } peers { permanent_uuid: "cfc89e8a34d34865b9f7956379d54d94" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38251 } } peers { permanent_uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36189 } }
I20260806 02:12:45.337932 23990 raft_consensus.cc:374] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:45.338001 23990 raft_consensus.cc:729] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfc89e8a34d34865b9f7956379d54d94, State: Initialized, Role: FOLLOWER
I20260806 02:12:45.337945 23989 consensus_queue.cc:260] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc04c752fbdb464cb95353b57c5a48d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 33065 } } peers { permanent_uuid: "cfc89e8a34d34865b9f7956379d54d94" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38251 } } peers { permanent_uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36189 } }
I20260806 02:12:45.338451 23990 consensus_queue.cc:260] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc04c752fbdb464cb95353b57c5a48d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 33065 } } peers { permanent_uuid: "cfc89e8a34d34865b9f7956379d54d94" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38251 } } peers { permanent_uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36189 } }
I20260806 02:12:45.338910 23989 ts_tablet_manager.cc:1428] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 02:12:45.339244 23971 heartbeater.cc:503] Master 127.18.70.126:37493 was elected leader, sending a full tablet report...
I20260806 02:12:45.339349 23990 ts_tablet_manager.cc:1428] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 02:12:45.339713 23830 heartbeater.cc:503] Master 127.18.70.126:37493 was elected leader, sending a full tablet report...
W20260806 02:12:45.350427 23831 tablet.cc:2369] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:12:45.392055 23996 raft_consensus.cc:482] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:45.392302 23996 raft_consensus.cc:504] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc04c752fbdb464cb95353b57c5a48d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 33065 } } peers { permanent_uuid: "cfc89e8a34d34865b9f7956379d54d94" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38251 } } peers { permanent_uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36189 } }
I20260806 02:12:45.393586 23996 leader_election.cc:288] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc04c752fbdb464cb95353b57c5a48d5 (127.18.70.65:33065), 8df8c6a5546d4cc599dccc5ba2c21e9b (127.18.70.67:36189)
I20260806 02:12:45.395714 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.393807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:33065 (local address 127.18.70.66:60655)
0806 02:12:45.394040 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:45.394057 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:45.394180 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:45.394511 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:45.394517 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:45.394536 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:45.395094 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:12:45.395096 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:45.395106 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:12:45.395129 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:45.395423 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:45.395426 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:45.395429 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 02:12:45.395519 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 02:12:45.395573 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260806 02:12:45.395943 23983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.393943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:60655 (local address 127.18.70.65:33065)
0806 02:12:45.394080 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:45.394085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:45.394161 (+ 76us) server_negotiation.cc:407] Connection header received
0806 02:12:45.394306 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:45.394311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:45.394379 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:45.394463 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:45.395253 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:45.395258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:45.395261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:45.395275 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:45.395298 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:45.395311 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:45.395442 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:45.395744 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 02:12:45.395813 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 02:12:45.396056 23987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.394359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:53903 (local address 127.18.70.67:36189)
0806 02:12:45.394498 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:45.394502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:45.394679 (+ 177us) server_negotiation.cc:407] Connection header received
0806 02:12:45.394762 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:45.394764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:45.394796 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:45.394861 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:45.395535 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:45.395537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:45.395539 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:45.395548 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:45.395611 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:45.395621 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:45.395702 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:45.395856 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 02:12:45.395904 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 02:12:45.396381 23997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.394157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:36189 (local address 127.18.70.66:53903)
0806 02:12:45.394581 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:45.394592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:45.394666 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:45.394904 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:45.394909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:45.394928 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:45.395405 (+ 477us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:12:45.395409 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:45.395418 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:12:45.395440 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:45.395705 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:45.395708 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:45.395711 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 02:12:45.396203 (+ 492us) client_negotiation.cc:240] Negotiation successful
0806 02:12:45.396243 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":141,"threads_started":1}
I20260806 02:12:45.396564 23641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66677b4942bc4e619f727a5656f2185e" candidate_uuid: "cfc89e8a34d34865b9f7956379d54d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc04c752fbdb464cb95353b57c5a48d5" is_pre_election: true
I20260806 02:12:45.396883 23641 raft_consensus.cc:2454] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfc89e8a34d34865b9f7956379d54d94 in term 0.
I20260806 02:12:45.397034 23920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66677b4942bc4e619f727a5656f2185e" candidate_uuid: "cfc89e8a34d34865b9f7956379d54d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b" is_pre_election: true
I20260806 02:12:45.397328 23920 raft_consensus.cc:2454] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfc89e8a34d34865b9f7956379d54d94 in term 0.
I20260806 02:12:45.397425 23710 leader_election.cc:302] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc04c752fbdb464cb95353b57c5a48d5, cfc89e8a34d34865b9f7956379d54d94; no voters:
I20260806 02:12:45.397725 23996 raft_consensus.cc:2781] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:45.397831 23996 raft_consensus.cc:482] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:45.397909 23996 raft_consensus.cc:3037] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:45.398993 23996 raft_consensus.cc:504] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc04c752fbdb464cb95353b57c5a48d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 33065 } } peers { permanent_uuid: "cfc89e8a34d34865b9f7956379d54d94" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38251 } } peers { permanent_uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36189 } }
I20260806 02:12:45.399335 23996 leader_election.cc:288] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [CANDIDATE]: Term 1 election: Requested vote from peers cc04c752fbdb464cb95353b57c5a48d5 (127.18.70.65:33065), 8df8c6a5546d4cc599dccc5ba2c21e9b (127.18.70.67:36189)
I20260806 02:12:45.399688 23641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66677b4942bc4e619f727a5656f2185e" candidate_uuid: "cfc89e8a34d34865b9f7956379d54d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc04c752fbdb464cb95353b57c5a48d5"
I20260806 02:12:45.399807 23641 raft_consensus.cc:3037] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:45.399809 23920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66677b4942bc4e619f727a5656f2185e" candidate_uuid: "cfc89e8a34d34865b9f7956379d54d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b"
I20260806 02:12:45.399938 23920 raft_consensus.cc:3037] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:45.400856 23641 raft_consensus.cc:2454] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfc89e8a34d34865b9f7956379d54d94 in term 1.
I20260806 02:12:45.401010 23920 raft_consensus.cc:2454] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfc89e8a34d34865b9f7956379d54d94 in term 1.
I20260806 02:12:45.401160 23710 leader_election.cc:302] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc04c752fbdb464cb95353b57c5a48d5, cfc89e8a34d34865b9f7956379d54d94; no voters:
I20260806 02:12:45.401352 23996 raft_consensus.cc:2781] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:45.401592 23996 raft_consensus.cc:686] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [term 1 LEADER]: Becoming Leader. State: Replica: cfc89e8a34d34865b9f7956379d54d94, State: Running, Role: LEADER
I20260806 02:12:45.401922 23996 consensus_queue.cc:237] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc04c752fbdb464cb95353b57c5a48d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 33065 } } peers { permanent_uuid: "cfc89e8a34d34865b9f7956379d54d94" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38251 } } peers { permanent_uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36189 } }
I20260806 02:12:45.405169 23493 catalog_manager.cc:5495] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to cfc89e8a34d34865b9f7956379d54d94 (127.18.70.66). New cstate: current_term: 1 leader_uuid: "cfc89e8a34d34865b9f7956379d54d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc04c752fbdb464cb95353b57c5a48d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 33065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfc89e8a34d34865b9f7956379d54d94" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36189 } health_report { overall_health: UNKNOWN } } }
W20260806 02:12:45.428898 23691 tablet.cc:2369] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:12:45.437099 23985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.435459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35080 (local address 127.18.70.66:38251)
0806 02:12:45.435602 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:45.435608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:45.435669 (+ 61us) server_negotiation.cc:407] Connection header received
0806 02:12:45.435912 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:45.435914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:45.435975 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:45.436066 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:45.436700 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:45.436704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:45.436706 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:45.436718 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:45.436738 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:45.436749 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:45.436847 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:45.436919 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 02:12:45.436959 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 02:12:45.442984 23920 raft_consensus.cc:1261] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b [term 1 FOLLOWER]: Refusing update from remote peer cfc89e8a34d34865b9f7956379d54d94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:45.442984 23641 raft_consensus.cc:1261] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5 [term 1 FOLLOWER]: Refusing update from remote peer cfc89e8a34d34865b9f7956379d54d94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:45.443756 23998 consensus_queue.cc:1035] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc04c752fbdb464cb95353b57c5a48d5" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:45.443958 23996 consensus_queue.cc:1035] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df8c6a5546d4cc599dccc5ba2c21e9b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 36189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:45.452204 24003 mvcc.cc:205] Tried to move back new op lower bound from 7315383768847302656 to 7315383768688742400. Current Snapshot: MvccSnapshot[applied={T|T < 7315383768847302656}]
I20260806 02:12:45.452430 24002 mvcc.cc:205] Tried to move back new op lower bound from 7315383768847302656 to 7315383768688742400. Current Snapshot: MvccSnapshot[applied={T|T < 7315383768847302656}]
I20260806 02:12:45.460680 23495 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:45.460856 23495 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:45.463337 23495 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c01ec59e92de4de883ce2389f2965938: Sending DeleteTablet for 3 replicas of tablet 66677b4942bc4e619f727a5656f2185e
I20260806 02:12:45.464228 23621 tablet_service.cc:1514] Processing DeleteTablet for tablet 66677b4942bc4e619f727a5656f2185e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:45 UTC) from {username='slave'} at 127.0.0.1:35276
I20260806 02:12:45.464277 23900 tablet_service.cc:1514] Processing DeleteTablet for tablet 66677b4942bc4e619f727a5656f2185e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:45 UTC) from {username='slave'} at 127.0.0.1:46164
I20260806 02:12:45.464778 24010 tablet_replica.cc:330] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5: stopping tablet replica
I20260806 02:12:45.464740 23758 tablet_service.cc:1514] Processing DeleteTablet for tablet 66677b4942bc4e619f727a5656f2185e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:45 UTC) from {username='slave'} at 127.0.0.1:35070
I20260806 02:12:45.465063 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 23557
I20260806 02:12:45.465095 24010 raft_consensus.cc:2229] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:45.465395 24010 raft_consensus.cc:2258] T 66677b4942bc4e619f727a5656f2185e P cc04c752fbdb464cb95353b57c5a48d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:45.465737 24011 tablet_replica.cc:330] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b: stopping tablet replica
I20260806 02:12:45.466101 24011 raft_consensus.cc:2229] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:45.466372 24011 raft_consensus.cc:2258] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:45.468140 24011 ts_tablet_manager.cc:1905] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:45.469218 24013 tablet_replica.cc:330] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94: stopping tablet replica
I20260806 02:12:45.469511 24013 raft_consensus.cc:2229] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:12:45.469888 24013 raft_consensus.cc:2258] T 66677b4942bc4e619f727a5656f2185e P cfc89e8a34d34865b9f7956379d54d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:45.471616 24011 ts_tablet_manager.cc:1918] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:45.471714 24011 log.cc:1196] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/66677b4942bc4e619f727a5656f2185e
I20260806 02:12:45.472060 24011 ts_tablet_manager.cc:1939] T 66677b4942bc4e619f727a5656f2185e P 8df8c6a5546d4cc599dccc5ba2c21e9b: Deleting consensus metadata
I20260806 02:12:45.473384 23480 catalog_manager.cc:4841] TS 8df8c6a5546d4cc599dccc5ba2c21e9b (127.18.70.67:36189): tablet 66677b4942bc4e619f727a5656f2185e (table test-table [id=42516aa6cfb34f9ea49700def1219a48]) successfully deleted
W20260806 02:12:45.475524 23480 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:33065 (error 108)
I20260806 02:12:45.476444 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 23693
I20260806 02:12:45.477216 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.476792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:33065 (local address 127.0.0.1:35290)
0806 02:12:45.476986 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:45.477093 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:33065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260806 02:12:45.477427 23480 catalog_manager.cc:4584] TS cc04c752fbdb464cb95353b57c5a48d5 (127.18.70.65:33065): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 66677b4942bc4e619f727a5656f2185e: Network error: Client connection negotiation failed: client connection to 127.18.70.65:33065: connect: Connection refused (error 111)
I20260806 02:12:45.483891 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 23833
I20260806 02:12:45.487093 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.484058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:38251 (local address 127.0.0.1:35092)
0806 02:12:45.486917 (+ 2859us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:45.486977 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.66:38251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2745}
I20260806 02:12:45.491968 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 23464
2026-08-06T02:12:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (998 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 02:12:45.515029 18713 mini_oidc.cc:152] Starting JWKS server
I20260806 02:12:45.516880 18713 webserver.cc:460] Webserver started at https://127.0.0.1:33855/ using document root <none> and password file <none>
I20260806 02:12:45.517064 18713 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 02:12:45.517356 18713 webserver.cc:460] Webserver started at http://127.0.0.1:33297/ using document root <none> and password file <none>
2026-08-06T02:12:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:45Z Disabled control of system clock
I20260806 02:12:45.527361 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:40511
--webserver_interface=127.18.70.126
--webserver_port=0
--builtin_ntp_servers=127.18.70.84:45367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:40511
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33855/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/testchainca.cert with env {}
W20260806 02:12:45.640431 24020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:45.640728 24020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:45.640770 24020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:45.644665 24020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:45.644716 24020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:45.644732 24020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:45.644750 24020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:45.644766 24020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 02:12:45.644783 24020 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 02:12:45.644801 24020 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33855/jwks.json
I20260806 02:12:45.650048 24020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:45367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:40511
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:40511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33855/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:45.651010 24020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:45.651965 24020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:45.660152 24026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:45.660161 24028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:45.660416 24025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:45.660461 24020 server_base.cc:1034] running on GCE node
I20260806 02:12:45.661119 24020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:45.662174 24020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:45.663374 24020 hybrid_clock.cc:648] HybridClock initialized: now 1785982365663346 us; error 39 us; skew 500 ppm
I20260806 02:12:45.665206 24020 webserver.cc:460] Webserver started at http://127.18.70.126:39331/ using document root <none> and password file <none>
I20260806 02:12:45.665788 24020 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:45.665849 24020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:45.666081 24020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:45.668222 24020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "16161b5b9028496eabfddb7a1c0eae0e"
format_stamp: "Formatted at 2026-08-06 02:12:45 on dist-test-slave-6k22"
server_key: "ac472ada728d22cbe61c728105bd7b5f"
server_key_iv: "1bdc85eb17a5761d37c561dd6c3d302e"
server_key_version: "encryptionkey@0"
I20260806 02:12:45.668788 24020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "16161b5b9028496eabfddb7a1c0eae0e"
format_stamp: "Formatted at 2026-08-06 02:12:45 on dist-test-slave-6k22"
server_key: "ac472ada728d22cbe61c728105bd7b5f"
server_key_iv: "1bdc85eb17a5761d37c561dd6c3d302e"
server_key_version: "encryptionkey@0"
I20260806 02:12:45.672640 24020 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 02:12:45.674998 24034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:45.676296 24020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:45.676416 24020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "16161b5b9028496eabfddb7a1c0eae0e"
format_stamp: "Formatted at 2026-08-06 02:12:45 on dist-test-slave-6k22"
server_key: "ac472ada728d22cbe61c728105bd7b5f"
server_key_iv: "1bdc85eb17a5761d37c561dd6c3d302e"
server_key_version: "encryptionkey@0"
I20260806 02:12:45.676519 24020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:45.688470 24020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:45.689039 24020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:45.689173 24020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:45.696120 24020 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:40511
I20260806 02:12:45.696132 24089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:40511 every 8 connection(s)
I20260806 02:12:45.697021 24020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:45.700199 24090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:45.703022 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 24020
I20260806 02:12:45.703162 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:45.703464 18713 external_mini_cluster.cc:1428] Setting key 866d00f058a708e1cc3658ab2f975175
I20260806 02:12:45.706018 24090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e: Bootstrap starting.
I20260806 02:12:45.708420 24090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:45.709267 24090 log.cc:824] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:45.711011 24090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e: No bootstrap required, opened a new log
I20260806 02:12:45.712478 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.704789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45272 (local address 127.18.70.126:40511)
0806 02:12:45.705392 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:45.705407 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:45.705449 (+ 42us) server_negotiation.cc:407] Connection header received
0806 02:12:45.706099 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:45.706125 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:45.706529 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:45.706807 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:45.707133 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:45.708132 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:45.709397 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:45.709859 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:45.710031 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:45.710845 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:45.710850 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:45.710862 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:45.711261 (+ 399us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:45.711314 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:45.711334 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:45.711453 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:45.711599 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 02:12:45.711848 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":131,"threads_started":1}
I20260806 02:12:45.714849 24090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16161b5b9028496eabfddb7a1c0eae0e" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40511 } }
I20260806 02:12:45.715116 24090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:45.715175 24090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16161b5b9028496eabfddb7a1c0eae0e, State: Initialized, Role: FOLLOWER
I20260806 02:12:45.715478 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:12:45.715649 24090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16161b5b9028496eabfddb7a1c0eae0e" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40511 } }
I20260806 02:12:45.715803 24090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:45.715867 24090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:45.715960 24090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:45.716979 24090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16161b5b9028496eabfddb7a1c0eae0e" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40511 } }
I20260806 02:12:45.717340 24090 leader_election.cc:302] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16161b5b9028496eabfddb7a1c0eae0e; no voters:
I20260806 02:12:45.717653 24090 leader_election.cc:288] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:45.717730 24096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:45.717980 24096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [term 1 LEADER]: Becoming Leader. State: Replica: 16161b5b9028496eabfddb7a1c0eae0e, State: Running, Role: LEADER
I20260806 02:12:45.718288 24096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16161b5b9028496eabfddb7a1c0eae0e" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40511 } }
I20260806 02:12:45.718976 24090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:45.720078 24097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16161b5b9028496eabfddb7a1c0eae0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16161b5b9028496eabfddb7a1c0eae0e" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40511 } } }
I20260806 02:12:45.720099 24098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16161b5b9028496eabfddb7a1c0eae0e. Latest consensus state: current_term: 1 leader_uuid: "16161b5b9028496eabfddb7a1c0eae0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16161b5b9028496eabfddb7a1c0eae0e" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40511 } } }
I20260806 02:12:45.720223 24097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:45.720223 24098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:45.720613 24105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:45.723826 24105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:45.728937 24105 catalog_manager.cc:1347] Generated new cluster ID: ec05a40d58784d8793b08717f595574a
I20260806 02:12:45.729035 24105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:45.738546 24105 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:45.739663 24105 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:45.748813 24105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16161b5b9028496eabfddb7a1c0eae0e: Generated new TSK 0
I20260806 02:12:45.749616 24105 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 02:12:45.775890 24123 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 02:12:45.776051 24123 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 02:12:45.776564 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:45.759724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45274 (local address 127.18.70.126:40511)
0806 02:12:45.759870 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:45.759875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:45.760077 (+ 202us) server_negotiation.cc:407] Connection header received
0806 02:12:45.760283 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:45.760285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:45.760328 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:45.760409 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0806 02:12:45.760721 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:45.761222 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:45.762397 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:45.762606 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:45.762687 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:45.763057 (+ 370us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 02:12:45.776247 (+ 13190us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 02:12:45.776465 (+ 218us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45274: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 02:12:45.776643 24094 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45274: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 02:12:45.778120 24122 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:12:45.759667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:40511 (local address 127.0.0.1:45274)
0806 02:12:45.759987 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:45.760003 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:45.760128 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:45.760474 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:45.760478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:45.760488 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0806 02:12:45.760606 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:45.760613 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:45.761345 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:45.761348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:45.762280 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:45.762292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:45.762722 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:45.762726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:45.762806 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:45.762882 (+ 76us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 02:12:45.776466 (+ 13584us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 02:12:45.776721 (+ 255us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.70.126:40511: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":116,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.70.126:40511: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 02:12:45.778882 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 24020
2026-08-06T02:12:45Z chronyd exiting
I20260806 02:12:45.818383 18713 test_util.cc:175] -----------------------------------------------
I20260806 02:12:45.818506 18713 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785982339772911-18713-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (305 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 02:12:45.819733 18713 mini_oidc.cc:152] Starting JWKS server
I20260806 02:12:45.820634 18713 webserver.cc:460] Webserver started at https://127.0.0.1:35705/ using document root <none> and password file <none>
I20260806 02:12:45.820782 18713 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 02:12:45.821048 18713 webserver.cc:460] Webserver started at http://127.0.0.1:40493/ using document root <none> and password file <none>
2026-08-06T02:12:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:45Z Disabled control of system clock
I20260806 02:12:45.830840 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:33969
--webserver_interface=127.18.70.126
--webserver_port=0
--builtin_ntp_servers=127.18.70.84:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:33969
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35705/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/testchainca.cert with env {}
W20260806 02:12:45.942827 24129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:45.943064 24129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:45.943104 24129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:45.946975 24129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:45.947041 24129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:45.947057 24129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:45.947072 24129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:45.947086 24129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 02:12:45.947101 24129 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 02:12:45.947114 24129 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35705/jwks.json
I20260806 02:12:45.952188 24129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:33969
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:33969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35705/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:45.953104 24129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:45.954387 24129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:45.962865 24137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:45.962859 24134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:45.962872 24135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:45.963001 24129 server_base.cc:1034] running on GCE node
I20260806 02:12:45.963780 24129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:45.964849 24129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:45.966006 24129 hybrid_clock.cc:648] HybridClock initialized: now 1785982365965988 us; error 30 us; skew 500 ppm
I20260806 02:12:45.967808 24129 webserver.cc:460] Webserver started at http://127.18.70.126:40805/ using document root <none> and password file <none>
I20260806 02:12:45.968369 24129 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:45.968431 24129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:45.968676 24129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:45.970626 24129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "d81b580036704dce9f8360d081e27719"
format_stamp: "Formatted at 2026-08-06 02:12:45 on dist-test-slave-6k22"
server_key: "6f3e5de077b5944b23bc7bf8955e180f"
server_key_iv: "0af2b060329de0bfce95c2e1a774cee3"
server_key_version: "encryptionkey@0"
I20260806 02:12:45.971170 24129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "d81b580036704dce9f8360d081e27719"
format_stamp: "Formatted at 2026-08-06 02:12:45 on dist-test-slave-6k22"
server_key: "6f3e5de077b5944b23bc7bf8955e180f"
server_key_iv: "0af2b060329de0bfce95c2e1a774cee3"
server_key_version: "encryptionkey@0"
I20260806 02:12:45.974910 24129 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 02:12:45.977285 24143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:45.978492 24129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:45.978631 24129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "d81b580036704dce9f8360d081e27719"
format_stamp: "Formatted at 2026-08-06 02:12:45 on dist-test-slave-6k22"
server_key: "6f3e5de077b5944b23bc7bf8955e180f"
server_key_iv: "0af2b060329de0bfce95c2e1a774cee3"
server_key_version: "encryptionkey@0"
I20260806 02:12:45.978742 24129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:45.993778 24129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:45.994431 24129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:45.994623 24129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:46.001415 24129 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:33969
I20260806 02:12:46.001430 24197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:33969 every 8 connection(s)
I20260806 02:12:46.002492 24129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:46.005837 24198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:46.006479 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 24129
I20260806 02:12:46.006604 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:46.006917 18713 external_mini_cluster.cc:1428] Setting key 451477ca5d9fbe61099651d2bf743225
I20260806 02:12:46.013283 24198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719: Bootstrap starting.
I20260806 02:12:46.015990 24198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:46.016849 24198 log.cc:824] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:46.017808 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:46.008642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53716 (local address 127.18.70.126:33969)
0806 02:12:46.009377 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:46.009399 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:46.009444 (+ 45us) server_negotiation.cc:407] Connection header received
0806 02:12:46.010281 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:46.010305 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:46.010791 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:46.011144 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:46.011609 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.013018 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.014222 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.014933 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.015084 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:46.015897 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:46.015904 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:46.015933 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:46.016399 (+ 466us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:46.016462 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:46.016487 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:46.016573 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:46.016686 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 02:12:46.016961 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":132,"threads_started":1}
I20260806 02:12:46.018865 24198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719: No bootstrap required, opened a new log
I20260806 02:12:46.020643 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:12:46.023118 24198 raft_consensus.cc:348] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81b580036704dce9f8360d081e27719" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33969 } }
I20260806 02:12:46.023319 24198 raft_consensus.cc:374] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:46.023353 24198 raft_consensus.cc:729] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d81b580036704dce9f8360d081e27719, State: Initialized, Role: FOLLOWER
I20260806 02:12:46.023759 24198 consensus_queue.cc:260] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81b580036704dce9f8360d081e27719" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33969 } }
I20260806 02:12:46.023880 24198 raft_consensus.cc:388] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:46.023924 24198 raft_consensus.cc:482] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:46.024026 24198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:46.024842 24198 raft_consensus.cc:504] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81b580036704dce9f8360d081e27719" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33969 } }
I20260806 02:12:46.025131 24198 leader_election.cc:302] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d81b580036704dce9f8360d081e27719; no voters:
I20260806 02:12:46.025374 24198 leader_election.cc:288] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:46.025501 24204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:46.025722 24204 raft_consensus.cc:686] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [term 1 LEADER]: Becoming Leader. State: Replica: d81b580036704dce9f8360d081e27719, State: Running, Role: LEADER
I20260806 02:12:46.026036 24204 consensus_queue.cc:237] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81b580036704dce9f8360d081e27719" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33969 } }
I20260806 02:12:46.026618 24198 sys_catalog.cc:564] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:46.027766 24206 sys_catalog.cc:455] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d81b580036704dce9f8360d081e27719. Latest consensus state: current_term: 1 leader_uuid: "d81b580036704dce9f8360d081e27719" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81b580036704dce9f8360d081e27719" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33969 } } }
I20260806 02:12:46.027768 24205 sys_catalog.cc:455] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d81b580036704dce9f8360d081e27719" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81b580036704dce9f8360d081e27719" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33969 } } }
I20260806 02:12:46.027906 24206 sys_catalog.cc:458] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:46.027931 24205 sys_catalog.cc:458] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:46.028306 24213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:46.031198 24213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:46.036024 24213 catalog_manager.cc:1347] Generated new cluster ID: 8eb67d2641c948eeb5c0813d34508c5d
I20260806 02:12:46.036100 24213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:46.069732 24213 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:46.071079 24213 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:46.079322 24213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d81b580036704dce9f8360d081e27719: Generated new TSK 0
I20260806 02:12:46.080142 24213 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 02:12:46.113833 24229 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 02:12:46.113917 24229 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 02:12:46.114480 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:46.091928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53726 (local address 127.18.70.126:33969)
0806 02:12:46.092054 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:46.092060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:46.092270 (+ 210us) server_negotiation.cc:407] Connection header received
0806 02:12:46.092413 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:46.092415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:46.092457 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:46.092536 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0806 02:12:46.092773 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.093269 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.094496 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.094712 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.094785 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:46.095095 (+ 310us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 02:12:46.114135 (+ 19040us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 02:12:46.114373 (+ 238us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53726: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 02:12:46.114560 24202 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53726: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 02:12:46.114665 24228 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:12:46.091875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:33969 (local address 127.0.0.1:53726)
0806 02:12:46.092178 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:46.092192 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:46.092304 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:46.092558 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:46.092561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:46.092571 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0806 02:12:46.092677 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:46.092683 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.093372 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.093378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:46.094385 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:46.094395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.094826 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.094830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:46.094917 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:46.094945 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 02:12:46.114325 (+ 19380us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 02:12:46.114484 (+ 159us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.70.126:33969: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":100,"threads_started":1}
I20260806 02:12:46.115279 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 24129
2026-08-06T02:12:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (337 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 02:12:46.157310 18713 mini_oidc.cc:152] Starting JWKS server
I20260806 02:12:46.158162 18713 webserver.cc:460] Webserver started at https://127.0.0.1:39043/ using document root <none> and password file <none>
I20260806 02:12:46.158308 18713 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 02:12:46.158635 18713 webserver.cc:460] Webserver started at http://127.0.0.1:45629/ using document root <none> and password file <none>
2026-08-06T02:12:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:46Z Disabled control of system clock
I20260806 02:12:46.168494 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:36619
--webserver_interface=127.18.70.126
--webserver_port=0
--builtin_ntp_servers=127.18.70.84:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:36619
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39043/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 02:12:46.282101 24235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:46.282433 24235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:46.282514 24235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:46.286538 24235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:46.286619 24235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:46.286650 24235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:46.286705 24235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:46.286741 24235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 02:12:46.286777 24235 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 02:12:46.286809 24235 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39043/jwks.json
I20260806 02:12:46.291468 24235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:36619
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:36619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39043/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:46.292590 24235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:46.294258 24235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:46.303195 24243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:46.303282 24235 server_base.cc:1034] running on GCE node
W20260806 02:12:46.303169 24241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:46.303188 24240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:46.304025 24235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:46.305089 24235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:46.306280 24235 hybrid_clock.cc:648] HybridClock initialized: now 1785982366306255 us; error 43 us; skew 500 ppm
I20260806 02:12:46.308090 24235 webserver.cc:460] Webserver started at http://127.18.70.126:38785/ using document root <none> and password file <none>
I20260806 02:12:46.308696 24235 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:46.308802 24235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:46.309046 24235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:46.311134 24235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "e82c08bfa6c64727b64cb1bc1eb12085"
format_stamp: "Formatted at 2026-08-06 02:12:46 on dist-test-slave-6k22"
server_key: "c751e5f492a5a78ac3ede2fef0844523"
server_key_iv: "ac0f4685921c5f169a6bbf4868bda743"
server_key_version: "encryptionkey@0"
I20260806 02:12:46.311744 24235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "e82c08bfa6c64727b64cb1bc1eb12085"
format_stamp: "Formatted at 2026-08-06 02:12:46 on dist-test-slave-6k22"
server_key: "c751e5f492a5a78ac3ede2fef0844523"
server_key_iv: "ac0f4685921c5f169a6bbf4868bda743"
server_key_version: "encryptionkey@0"
I20260806 02:12:46.315299 24235 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:12:46.317692 24249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:46.318905 24235 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 02:12:46.319047 24235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "e82c08bfa6c64727b64cb1bc1eb12085"
format_stamp: "Formatted at 2026-08-06 02:12:46 on dist-test-slave-6k22"
server_key: "c751e5f492a5a78ac3ede2fef0844523"
server_key_iv: "ac0f4685921c5f169a6bbf4868bda743"
server_key_version: "encryptionkey@0"
I20260806 02:12:46.319159 24235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:46.335407 24235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:46.336120 24235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:46.336324 24235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:46.343453 24235 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:36619
I20260806 02:12:46.343523 24304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:36619 every 8 connection(s)
I20260806 02:12:46.344506 24235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:46.347651 24305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:46.353118 24305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085: Bootstrap starting.
I20260806 02:12:46.354271 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 24235
I20260806 02:12:46.354401 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:46.354686 18713 external_mini_cluster.cc:1428] Setting key ed7bcfdeb88f8da0e9c7c8d4daae6f09
I20260806 02:12:46.356149 24305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:46.356901 24305 log.cc:824] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:46.358927 24305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085: No bootstrap required, opened a new log
I20260806 02:12:46.362620 24305 raft_consensus.cc:348] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82c08bfa6c64727b64cb1bc1eb12085" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 36619 } }
I20260806 02:12:46.362856 24305 raft_consensus.cc:374] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:46.362926 24305 raft_consensus.cc:729] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82c08bfa6c64727b64cb1bc1eb12085, State: Initialized, Role: FOLLOWER
I20260806 02:12:46.363399 24305 consensus_queue.cc:260] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82c08bfa6c64727b64cb1bc1eb12085" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 36619 } }
I20260806 02:12:46.363551 24305 raft_consensus.cc:388] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:46.363655 24305 raft_consensus.cc:482] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:46.363756 24305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:46.363942 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:46.355768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.18.70.126:36619)
0806 02:12:46.356460 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:46.356474 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:46.356510 (+ 36us) server_negotiation.cc:407] Connection header received
0806 02:12:46.357173 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:46.357192 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:46.357561 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:46.357904 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:46.358345 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.359445 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.360388 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.360877 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.361063 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:46.362011 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:46.362016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:46.362030 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:46.362422 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:46.362493 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:46.362513 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:46.362624 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:46.362777 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 02:12:46.363063 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":130,"threads_started":1}
I20260806 02:12:46.364885 24305 raft_consensus.cc:504] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82c08bfa6c64727b64cb1bc1eb12085" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 36619 } }
I20260806 02:12:46.365218 24305 leader_election.cc:302] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e82c08bfa6c64727b64cb1bc1eb12085; no voters:
I20260806 02:12:46.365509 24305 leader_election.cc:288] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:46.365587 24310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:46.365798 24310 raft_consensus.cc:686] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [term 1 LEADER]: Becoming Leader. State: Replica: e82c08bfa6c64727b64cb1bc1eb12085, State: Running, Role: LEADER
I20260806 02:12:46.366077 24310 consensus_queue.cc:237] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82c08bfa6c64727b64cb1bc1eb12085" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 36619 } }
I20260806 02:12:46.366688 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:12:46.366886 24305 sys_catalog.cc:564] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:46.367857 24312 sys_catalog.cc:455] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e82c08bfa6c64727b64cb1bc1eb12085. Latest consensus state: current_term: 1 leader_uuid: "e82c08bfa6c64727b64cb1bc1eb12085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82c08bfa6c64727b64cb1bc1eb12085" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 36619 } } }
I20260806 02:12:46.368525 24312 sys_catalog.cc:458] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:46.368836 24311 sys_catalog.cc:455] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e82c08bfa6c64727b64cb1bc1eb12085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82c08bfa6c64727b64cb1bc1eb12085" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 36619 } } }
I20260806 02:12:46.369283 24311 sys_catalog.cc:458] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085 [sys.catalog]: This master's current role is: LEADER
W20260806 02:12:46.371348 24325 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:12:46.371446 24325 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:12:46.371527 24324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:46.372270 24324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:46.377722 24324 catalog_manager.cc:1347] Generated new cluster ID: 2c1eea386cb04c4ab630ce7249aa25c1
I20260806 02:12:46.377795 24324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:46.395534 24324 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:46.396751 24324 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:46.406596 24324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e82c08bfa6c64727b64cb1bc1eb12085: Generated new TSK 0
I20260806 02:12:46.407392 24324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:12:46.424098 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:46.418798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60104 (local address 127.18.70.126:36619)
0806 02:12:46.418936 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:46.418941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:46.419103 (+ 162us) server_negotiation.cc:407] Connection header received
0806 02:12:46.419258 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:46.419260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:46.419306 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:46.419393 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0806 02:12:46.419693 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.420529 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.421417 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.421770 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.421879 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:46.422227 (+ 348us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 02:12:46.423711 (+ 1484us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39043/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 02:12:46.423941 (+ 230us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60104: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39043/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 02:12:46.424165 24308 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60104: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39043/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 02:12:46.425829 24334 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:12:46.418715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:36619 (local address 127.0.0.1:60104)
0806 02:12:46.419011 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:46.419027 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:46.419141 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:46.419419 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:46.419422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:46.419430 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 02:12:46.419547 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:46.419553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.420646 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.420649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:46.421302 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:46.421309 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.421889 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.421891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:46.422022 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:46.422045 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 02:12:46.423876 (+ 1831us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39043/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 02:12:46.425668 (+ 1792us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.70.126:36619: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39043/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":1616,"thread_start_us":117,"threads_started":1}
I20260806 02:12:46.426291 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 24235
E20260806 02:12:46.434906 24335 webserver.cc:533] Webserver: sslize failed:
E20260806 02:12:46.434993 24335 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T02:12:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (310 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T02:12:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:46Z Disabled control of system clock
I20260806 02:12:46.476853 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:34815
--webserver_interface=127.18.70.126
--webserver_port=0
--builtin_ntp_servers=127.18.70.84:34569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:34815
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/testchainca.cert with env {}
W20260806 02:12:46.591774 24340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:46.592049 24340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:46.592088 24340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:46.595963 24340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:46.596014 24340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:46.596030 24340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:46.596046 24340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:46.596061 24340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 02:12:46.596077 24340 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 02:12:46.596091 24340 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 02:12:46.601193 24340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:34569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:34815
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:34815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:46.602227 24340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:46.603183 24340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:46.611972 24346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:46.611981 24345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:46.611972 24348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:46.612282 24340 server_base.cc:1034] running on GCE node
I20260806 02:12:46.612787 24340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:46.614135 24340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:46.615406 24340 hybrid_clock.cc:648] HybridClock initialized: now 1785982366615388 us; error 44 us; skew 500 ppm
I20260806 02:12:46.617249 24340 webserver.cc:460] Webserver started at http://127.18.70.126:33063/ using document root <none> and password file <none>
I20260806 02:12:46.618038 24340 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:46.618120 24340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:46.618335 24340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:46.620519 24340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "a5b510ae806f413fb32a3342a9dc526b"
format_stamp: "Formatted at 2026-08-06 02:12:46 on dist-test-slave-6k22"
server_key: "df78aac33a4f9fee5cf6cb58625dd933"
server_key_iv: "14069a957b3f665bcf83de0522d1a867"
server_key_version: "encryptionkey@0"
I20260806 02:12:46.621091 24340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "a5b510ae806f413fb32a3342a9dc526b"
format_stamp: "Formatted at 2026-08-06 02:12:46 on dist-test-slave-6k22"
server_key: "df78aac33a4f9fee5cf6cb58625dd933"
server_key_iv: "14069a957b3f665bcf83de0522d1a867"
server_key_version: "encryptionkey@0"
I20260806 02:12:46.624794 24340 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:12:46.627112 24354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:46.628336 24340 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:46.628476 24340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "a5b510ae806f413fb32a3342a9dc526b"
format_stamp: "Formatted at 2026-08-06 02:12:46 on dist-test-slave-6k22"
server_key: "df78aac33a4f9fee5cf6cb58625dd933"
server_key_iv: "14069a957b3f665bcf83de0522d1a867"
server_key_version: "encryptionkey@0"
I20260806 02:12:46.628587 24340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:46.661783 24340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:46.662554 24340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:46.662864 24340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:46.670315 24340 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:34815
I20260806 02:12:46.670325 24409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:34815 every 8 connection(s)
I20260806 02:12:46.671449 24340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:46.673156 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 24340
I20260806 02:12:46.673292 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:46.673606 18713 external_mini_cluster.cc:1428] Setting key f55280e91065b5c476dce1724877f319
I20260806 02:12:46.675076 24410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:46.682018 24410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b: Bootstrap starting.
I20260806 02:12:46.684254 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:46.675048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46140 (local address 127.18.70.126:34815)
0806 02:12:46.675836 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:46.675857 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:46.675902 (+ 45us) server_negotiation.cc:407] Connection header received
0806 02:12:46.676800 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:46.676825 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:46.677330 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:46.677683 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:46.678144 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.679685 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.680519 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.681216 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.681388 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:46.682192 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:46.682198 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:46.682212 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:12:46.682685 (+ 473us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:46.682751 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:12:46.682778 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:46.682876 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:46.683018 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 02:12:46.683307 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":173,"threads_started":1}
I20260806 02:12:46.684739 24410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:46.685559 24410 log.cc:824] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:46.687273 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:12:46.688014 24410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b: No bootstrap required, opened a new log
I20260806 02:12:46.691733 24410 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5b510ae806f413fb32a3342a9dc526b" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 34815 } }
I20260806 02:12:46.691951 24410 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:46.692039 24410 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5b510ae806f413fb32a3342a9dc526b, State: Initialized, Role: FOLLOWER
I20260806 02:12:46.692515 24410 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5b510ae806f413fb32a3342a9dc526b" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 34815 } }
I20260806 02:12:46.692654 24410 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:46.692716 24410 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:46.692821 24410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:46.693743 24410 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5b510ae806f413fb32a3342a9dc526b" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 34815 } }
I20260806 02:12:46.694090 24410 leader_election.cc:302] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5b510ae806f413fb32a3342a9dc526b; no voters:
I20260806 02:12:46.694420 24410 leader_election.cc:288] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:46.694563 24415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:46.694815 24415 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [term 1 LEADER]: Becoming Leader. State: Replica: a5b510ae806f413fb32a3342a9dc526b, State: Running, Role: LEADER
I20260806 02:12:46.695106 24415 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5b510ae806f413fb32a3342a9dc526b" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 34815 } }
I20260806 02:12:46.695741 24410 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:46.696923 24417 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5b510ae806f413fb32a3342a9dc526b. Latest consensus state: current_term: 1 leader_uuid: "a5b510ae806f413fb32a3342a9dc526b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5b510ae806f413fb32a3342a9dc526b" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 34815 } } }
I20260806 02:12:46.697069 24417 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:46.697387 24416 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5b510ae806f413fb32a3342a9dc526b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5b510ae806f413fb32a3342a9dc526b" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 34815 } } }
I20260806 02:12:46.697507 24416 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:46.697575 24424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:46.700445 24424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:46.705399 24424 catalog_manager.cc:1347] Generated new cluster ID: e7088db98c2f40f38061d220d471a4e6
I20260806 02:12:46.705457 24424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:46.736187 24424 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:46.737552 24424 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:46.754357 24424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5b510ae806f413fb32a3342a9dc526b: Generated new TSK 0
I20260806 02:12:46.755221 24424 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:12:47.248750 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:46.771047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.18.70.126:34815)
0806 02:12:46.771223 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:46.771227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:46.771379 (+ 152us) server_negotiation.cc:407] Connection header received
0806 02:12:46.771538 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:46.771540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:46.771633 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:46.771726 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0806 02:12:46.772009 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.772503 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.773766 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.773970 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.774045 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:46.774399 (+ 354us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 02:12:47.248206 (+473807us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 02:12:47.248588 (+ 382us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46144: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 02:12:47.248855 24413 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46144: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 02:12:47.248895 24439 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:12:46.770958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:34815 (local address 127.0.0.1:46144)
0806 02:12:46.771287 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:46.771302 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:46.771417 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:46.771752 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:46.771755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:46.771763 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 02:12:46.771898 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:46.771905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.772636 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.772640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:46.773648 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:46.773660 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:46.774096 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:46.774100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:46.774200 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:46.774229 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 02:12:47.248482 (+474253us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 02:12:47.248688 (+ 206us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.70.126:34815: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":115,"threads_started":1}
I20260806 02:12:47.249842 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 24340
2026-08-06T02:12:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (805 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (133 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (131 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:12:48 dist-test-slave-6k22 krb5kdc[24452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:12:48 dist-test-slave-6k22 krb5kdc[24452](info): set up 2 sockets
Aug 06 02:12:48 dist-test-slave-6k22 krb5kdc[24452](info): commencing operation
krb5kdc: starting...
W20260806 02:12:49.782145 18713 mini_kdc.cc:121] Time spent starting KDC: real 2.243s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:12:49 dist-test-slave-6k22 krb5kdc[24452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:12:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:12:49.904179 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:42119
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:35333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:42119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:50.019196 24467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:50.019510 24467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:50.019645 24467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:50.023541 24467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:50.023656 24467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:50.023728 24467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:50.023777 24467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:50.023816 24467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:50.028930 24467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:35333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:42119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:42119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:50.029989 24467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:50.031015 24467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:50.039652 24472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:50.039633 24473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:50.039731 24467 server_base.cc:1034] running on GCE node
W20260806 02:12:50.039623 24475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:50.040421 24467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:50.041606 24467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:50.042784 24467 hybrid_clock.cc:648] HybridClock initialized: now 1785982370042755 us; error 42 us; skew 500 ppm
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:50.048817 24467 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:12:50.049638 24467 webserver.cc:460] Webserver started at http://127.18.70.126:44389/ using document root <none> and password file <none>
I20260806 02:12:50.050205 24467 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:50.050292 24467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:50.050519 24467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:50.052629 24467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "01ae8a19494249fcb19a2842eaba2c6e"
format_stamp: "Formatted at 2026-08-06 02:12:50 on dist-test-slave-6k22"
server_key: "e345906f6ce04d9552cdfbb9c7f9188d"
server_key_iv: "049606aa856deef7aa26e796311629f8"
server_key_version: "encryptionkey@0"
I20260806 02:12:50.053218 24467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "01ae8a19494249fcb19a2842eaba2c6e"
format_stamp: "Formatted at 2026-08-06 02:12:50 on dist-test-slave-6k22"
server_key: "e345906f6ce04d9552cdfbb9c7f9188d"
server_key_iv: "049606aa856deef7aa26e796311629f8"
server_key_version: "encryptionkey@0"
I20260806 02:12:50.056855 24467 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:12:50.059123 24482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:50.060407 24467 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:12:50.060526 24467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "01ae8a19494249fcb19a2842eaba2c6e"
format_stamp: "Formatted at 2026-08-06 02:12:50 on dist-test-slave-6k22"
server_key: "e345906f6ce04d9552cdfbb9c7f9188d"
server_key_iv: "049606aa856deef7aa26e796311629f8"
server_key_version: "encryptionkey@0"
I20260806 02:12:50.060621 24467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:50.079874 24467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:50.082419 24467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:50.082553 24467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:50.089331 24467 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:42119
I20260806 02:12:50.089339 24537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:42119 every 8 connection(s)
I20260806 02:12:50.090345 24467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:50.093102 24538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:50.098398 24538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e: Bootstrap starting.
I20260806 02:12:50.100420 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 24467
I20260806 02:12:50.100515 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:50.100804 18713 external_mini_cluster.cc:1428] Setting key c96fba4546ca67bf78e7d193edd332a7
I20260806 02:12:50.100893 24538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:50.101766 24538 log.cc:824] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:50.103673 24538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e: No bootstrap required, opened a new log
I20260806 02:12:50.107643 24538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ae8a19494249fcb19a2842eaba2c6e" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42119 } }
I20260806 02:12:50.107923 24538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:50.108002 24538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ae8a19494249fcb19a2842eaba2c6e, State: Initialized, Role: FOLLOWER
I20260806 02:12:50.108529 24538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ae8a19494249fcb19a2842eaba2c6e" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42119 } }
I20260806 02:12:50.108707 24538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:50.108788 24538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:50.108875 24538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [term 0 FOLLOWER]: Advancing to term 1
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:50.109756 24538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ae8a19494249fcb19a2842eaba2c6e" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42119 } }
I20260806 02:12:50.110080 24538 leader_election.cc:302] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01ae8a19494249fcb19a2842eaba2c6e; no voters:
I20260806 02:12:50.110370 24538 leader_election.cc:288] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:50.110836 24543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:50.111392 24543 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [term 1 LEADER]: Becoming Leader. State: Replica: 01ae8a19494249fcb19a2842eaba2c6e, State: Running, Role: LEADER
I20260806 02:12:50.111750 24538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:50.111755 24543 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ae8a19494249fcb19a2842eaba2c6e" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42119 } }
I20260806 02:12:50.113580 24545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01ae8a19494249fcb19a2842eaba2c6e. Latest consensus state: current_term: 1 leader_uuid: "01ae8a19494249fcb19a2842eaba2c6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ae8a19494249fcb19a2842eaba2c6e" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42119 } } }
I20260806 02:12:50.113711 24545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:50.114029 24544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01ae8a19494249fcb19a2842eaba2c6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ae8a19494249fcb19a2842eaba2c6e" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42119 } } }
I20260806 02:12:50.114346 24544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:50.115960 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:50.101911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50218 (local address 127.18.70.126:42119)
0806 02:12:50.102346 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:50.102361 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:50.102394 (+ 33us) server_negotiation.cc:407] Connection header received
0806 02:12:50.102989 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:50.103017 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:50.103345 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:50.103667 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:50.104570 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.105531 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.106764 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.107188 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.107305 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:50.110135 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:50.110141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:50.110153 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:50.110179 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:50.112454 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.112968 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.112976 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.112983 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.113125 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.113483 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.113486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.113488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.114040 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:50.114254 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:50.114730 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 02:12:50.115031 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"spinlock_wait_cycles":3968,"thread_start_us":149,"threads_started":1}
W20260806 02:12:50.116948 24560 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:12:50.117002 24560 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:12:50.117092 24559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:50.117887 24559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:50.122807 24559 catalog_manager.cc:1347] Generated new cluster ID: e151a55d57a940f8b2c09cbfdef50d9d
I20260806 02:12:50.122869 24559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:50.140185 24559 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:50.141397 24559 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:50.148272 24559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01ae8a19494249fcb19a2842eaba2c6e: Generated new TSK 0
I20260806 02:12:50.148988 24559 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:12:50.178424 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42119
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:35333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:50.294435 24566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:50.294739 24566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:50.294821 24566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:50.299698 24566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:50.299809 24566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:50.299911 24566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:12:50.305495 24566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:35333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:50.306582 24566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:50.307735 24566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:50.317971 24571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:50.318048 24566 server_base.cc:1034] running on GCE node
W20260806 02:12:50.317953 24572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:50.317931 24574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:50.318596 24566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:50.319252 24566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:50.320433 24566 hybrid_clock.cc:648] HybridClock initialized: now 1785982370320417 us; error 26 us; skew 500 ppm
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:50.323048 24566 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:12:50.323897 24566 webserver.cc:460] Webserver started at http://127.18.70.65:40325/ using document root <none> and password file <none>
I20260806 02:12:50.324424 24566 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:50.324512 24566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:50.324752 24566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:50.326710 24566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "9459e517810845a2ba438823366c8f0b"
format_stamp: "Formatted at 2026-08-06 02:12:50 on dist-test-slave-6k22"
server_key: "770f84df730e00ff6f4e3aa6a2428bff"
server_key_iv: "ab02d266857c8bc226476b127ba95386"
server_key_version: "encryptionkey@0"
I20260806 02:12:50.327252 24566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "9459e517810845a2ba438823366c8f0b"
format_stamp: "Formatted at 2026-08-06 02:12:50 on dist-test-slave-6k22"
server_key: "770f84df730e00ff6f4e3aa6a2428bff"
server_key_iv: "ab02d266857c8bc226476b127ba95386"
server_key_version: "encryptionkey@0"
I20260806 02:12:50.330786 24566 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 02:12:50.333057 24582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:50.334162 24566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:50.334285 24566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "9459e517810845a2ba438823366c8f0b"
format_stamp: "Formatted at 2026-08-06 02:12:50 on dist-test-slave-6k22"
server_key: "770f84df730e00ff6f4e3aa6a2428bff"
server_key_iv: "ab02d266857c8bc226476b127ba95386"
server_key_version: "encryptionkey@0"
I20260806 02:12:50.334384 24566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:50.352566 24566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:50.355779 24566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:50.355965 24566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:50.356510 24566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:50.357276 24566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:50.357338 24566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:50.357403 24566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:50.357448 24566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:50.366305 24566 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:32993
I20260806 02:12:50.366344 24699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:32993 every 8 connection(s)
I20260806 02:12:50.367151 24566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:12:50.375967 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 24566
I20260806 02:12:50.376089 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:12:50.376364 18713 external_mini_cluster.cc:1428] Setting key 5d25aef559242ad54564108c8868a1d5
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:50.380605 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:50.368929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:43107 (local address 127.18.70.126:42119)
0806 02:12:50.369069 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:50.369074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:50.369957 (+ 883us) server_negotiation.cc:407] Connection header received
0806 02:12:50.371271 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:50.371275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:50.371323 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:50.371396 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:50.372576 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.373094 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.374168 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.374376 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.374447 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:50.376993 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:50.376997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:50.376999 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:50.377022 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:50.378566 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.379069 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.379073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.379075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.379155 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.379493 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.379497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.379499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.379740 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:50.379846 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:50.380332 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 02:12:50.380451 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:12:50.381428 24702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:50.369372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42119 (local address 127.18.70.65:43107)
0806 02:12:50.369812 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:50.369856 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:50.371084 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:50.371574 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:50.371580 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:50.371954 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:50.372440 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:50.372452 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.373211 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.373213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:50.374071 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:50.374079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.374500 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.374502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:50.374661 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:50.375196 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:50.375211 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:50.376826 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:50.378685 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:50.378691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:50.378705 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:50.378965 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:50.379246 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:50.379250 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:50.379252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:50.379398 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:50.379846 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:50.379854 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:50.380095 (+ 241us) client_negotiation.cc:769] Sending connection context
0806 02:12:50.380283 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 02:12:50.380611 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":109,"threads_started":1}
I20260806 02:12:50.382656 24700 heartbeater.cc:347] Connected to a master server at 127.18.70.126:42119
I20260806 02:12:50.382913 24700 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:50.383394 24700 heartbeater.cc:511] Master 127.18.70.126:42119 requested a full tablet report, sending...
I20260806 02:12:50.385186 24500 ts_manager.cc:194] Registered new tserver with Master: 9459e517810845a2ba438823366c8f0b (127.18.70.65:32993)
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
I20260806 02:12:50.386848 24500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:43107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:12:50.404633 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42119
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:35333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:50.515134 24705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:50.515364 24705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:50.515440 24705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:50.519420 24705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:50.519474 24705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:50.519560 24705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:12:50.524592 24705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:35333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:50.525514 24705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:50.526463 24705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:50.535604 24713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:50.535619 24711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:50.535612 24710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:50.535697 24705 server_base.cc:1034] running on GCE node
I20260806 02:12:50.536223 24705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:50.536832 24705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:50.538064 24705 hybrid_clock.cc:648] HybridClock initialized: now 1785982370538044 us; error 28 us; skew 500 ppm
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:50.540899 24705 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:12:50.541684 24705 webserver.cc:460] Webserver started at http://127.18.70.66:33955/ using document root <none> and password file <none>
I20260806 02:12:50.542213 24705 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:50.542284 24705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:50.542510 24705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:50.544540 24705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "5b680270ca48429bb8edec438dca98b8"
format_stamp: "Formatted at 2026-08-06 02:12:50 on dist-test-slave-6k22"
server_key: "392e7dfe801a716b1098ba5eecf62d7f"
server_key_iv: "94af29244342995df66f122fe9eb09c0"
server_key_version: "encryptionkey@0"
I20260806 02:12:50.545075 24705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "5b680270ca48429bb8edec438dca98b8"
format_stamp: "Formatted at 2026-08-06 02:12:50 on dist-test-slave-6k22"
server_key: "392e7dfe801a716b1098ba5eecf62d7f"
server_key_iv: "94af29244342995df66f122fe9eb09c0"
server_key_version: "encryptionkey@0"
I20260806 02:12:50.548789 24705 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:12:50.551076 24721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:50.552290 24705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:12:50.552416 24705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "5b680270ca48429bb8edec438dca98b8"
format_stamp: "Formatted at 2026-08-06 02:12:50 on dist-test-slave-6k22"
server_key: "392e7dfe801a716b1098ba5eecf62d7f"
server_key_iv: "94af29244342995df66f122fe9eb09c0"
server_key_version: "encryptionkey@0"
I20260806 02:12:50.552518 24705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:50.566174 24705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:50.569345 24705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:50.569547 24705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:50.570137 24705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:50.570945 24705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:50.571012 24705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:50.571089 24705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:50.571133 24705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:50.580313 24705 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:38149
I20260806 02:12:50.580343 24841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:38149 every 8 connection(s)
I20260806 02:12:50.581188 24705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:50.590615 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 24705
I20260806 02:12:50.590742 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:12:50.591001 18713 external_mini_cluster.cc:1428] Setting key 130457d4aa305b413ab29074c6dc0755
I20260806 02:12:50.594632 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:50.582929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:47951 (local address 127.18.70.126:42119)
0806 02:12:50.583055 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:50.583059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:50.583883 (+ 824us) server_negotiation.cc:407] Connection header received
0806 02:12:50.585226 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:50.585229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:50.585286 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:50.585389 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:50.586554 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.587117 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.588334 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.588560 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.588671 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:50.591031 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:50.591035 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:50.591037 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:50.591059 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:50.592611 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.593123 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.593128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.593130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.593233 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.593566 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.593571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.593573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.593752 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:50.593851 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:50.594332 (+ 481us) server_negotiation.cc:299] Negotiation successful
0806 02:12:50.594448 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 02:12:50.595476 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:50.583249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42119 (local address 127.18.70.66:47951)
0806 02:12:50.583736 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:50.583781 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:50.585017 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:50.585559 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:50.585566 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:50.585934 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:50.586409 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:50.586422 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.587244 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.587247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:50.588200 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:50.588211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.588708 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.588712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:50.588877 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:50.589392 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:50.589408 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:50.590866 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:50.592733 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:50.592739 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:50.592761 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:50.593016 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:50.593333 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:50.593336 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:50.593338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:50.593474 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:50.593865 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:50.593872 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:50.594103 (+ 231us) client_negotiation.cc:769] Sending connection context
0806 02:12:50.594282 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 02:12:50.594606 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":115,"threads_started":1}
I20260806 02:12:50.596628 24842 heartbeater.cc:347] Connected to a master server at 127.18.70.126:42119
I20260806 02:12:50.596881 24842 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:50.597484 24842 heartbeater.cc:511] Master 127.18.70.126:42119 requested a full tablet report, sending...
I20260806 02:12:50.598675 24500 ts_manager.cc:194] Registered new tserver with Master: 5b680270ca48429bb8edec438dca98b8 (127.18.70.66:38149)
I20260806 02:12:50.599545 24500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:47951
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:12:50.618930 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42119
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:35333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:50.728561 24847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:50.728813 24847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:50.728855 24847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:50.732752 24847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:50.732798 24847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:50.732854 24847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:12:50.737913 24847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:35333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:50.738977 24847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:50.739950 24847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:50.748936 24855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:50.748950 24852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:50.748948 24853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:50.749307 24847 server_base.cc:1034] running on GCE node
I20260806 02:12:50.749673 24847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:50.750295 24847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:50.751477 24847 hybrid_clock.cc:648] HybridClock initialized: now 1785982370751463 us; error 26 us; skew 500 ppm
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:50.754156 24847 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:12:50.754940 24847 webserver.cc:460] Webserver started at http://127.18.70.67:44863/ using document root <none> and password file <none>
I20260806 02:12:50.755492 24847 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:50.755601 24847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:50.755852 24847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:50.757768 24847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "edf731b4fa9142c68091757b05d0f8ac"
format_stamp: "Formatted at 2026-08-06 02:12:50 on dist-test-slave-6k22"
server_key: "357e0d7a85932f1eabdd396450069942"
server_key_iv: "bbac8f64a2eabae3218a9b9a64639ebb"
server_key_version: "encryptionkey@0"
I20260806 02:12:50.758301 24847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "edf731b4fa9142c68091757b05d0f8ac"
format_stamp: "Formatted at 2026-08-06 02:12:50 on dist-test-slave-6k22"
server_key: "357e0d7a85932f1eabdd396450069942"
server_key_iv: "bbac8f64a2eabae3218a9b9a64639ebb"
server_key_version: "encryptionkey@0"
I20260806 02:12:50.761861 24847 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:12:50.764204 24862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:50.765355 24847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:50.765482 24847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "edf731b4fa9142c68091757b05d0f8ac"
format_stamp: "Formatted at 2026-08-06 02:12:50 on dist-test-slave-6k22"
server_key: "357e0d7a85932f1eabdd396450069942"
server_key_iv: "bbac8f64a2eabae3218a9b9a64639ebb"
server_key_version: "encryptionkey@0"
I20260806 02:12:50.765584 24847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:50.788116 24847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:50.790800 24847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:50.790995 24847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:50.791632 24847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:50.792423 24847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:50.792488 24847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:50.792555 24847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:50.792605 24847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:50.801767 24847 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:33135
I20260806 02:12:50.801819 24978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:33135 every 8 connection(s)
I20260806 02:12:50.802659 24847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:12:50.804806 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 24847
I20260806 02:12:50.804943 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:12:50.805212 18713 external_mini_cluster.cc:1428] Setting key 1f542750afb9053481f7134e7a2cb368
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:12:50.815464 24542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:50.804465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:46197 (local address 127.18.70.126:42119)
0806 02:12:50.804648 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:50.804653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:50.805557 (+ 904us) server_negotiation.cc:407] Connection header received
0806 02:12:50.806906 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:50.806909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:50.806958 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:50.807035 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:50.808234 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.808748 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.809828 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.810044 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.810116 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:50.812375 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:50.812378 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:50.812380 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:50.812403 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:50.813674 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.814192 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.814194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.814195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.814277 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.814578 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.814580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.814581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.814750 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:50.814826 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:50.815244 (+ 418us) server_negotiation.cc:299] Negotiation successful
0806 02:12:50.815345 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 02:12:50.816290 24981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:50.804907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42119 (local address 127.18.70.67:46197)
0806 02:12:50.805407 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:50.805459 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:50.806725 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:50.807232 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:50.807239 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:50.807618 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:50.808103 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:50.808116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.808852 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.808854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:50.809698 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:50.809707 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.810161 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.810164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:50.810310 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:50.810774 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:50.810790 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:50.812228 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:50.813789 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:50.813794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:50.813805 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:50.814093 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:50.814353 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:50.814355 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:50.814357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:50.814484 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:50.814836 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:50.814841 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:50.815052 (+ 211us) client_negotiation.cc:769] Sending connection context
0806 02:12:50.815201 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 02:12:50.815439 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":127,"threads_started":1}
I20260806 02:12:50.817394 24979 heartbeater.cc:347] Connected to a master server at 127.18.70.126:42119
I20260806 02:12:50.817643 24979 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:50.818259 24979 heartbeater.cc:511] Master 127.18.70.126:42119 requested a full tablet report, sending...
I20260806 02:12:50.819305 24500 ts_manager.cc:194] Registered new tserver with Master: edf731b4fa9142c68091757b05d0f8ac (127.18.70.67:33135)
I20260806 02:12:50.819893 24500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:46197
I20260806 02:12:50.830749 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/test-admin.keytab.
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:50.848646 18713 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 02:12:50.851403 24986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:12:50.849970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34744 (local address 127.18.70.65:32993)
0806 02:12:50.850303 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:50.850339 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:50.850362 (+ 23us) server_negotiation.cc:407] Connection header received
0806 02:12:50.850411 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:50.850416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:50.850601 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:50.850717 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:50.851191 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34744: BlockingRecv error: recv got EOF from 127.0.0.1:34744 (error 108)
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":141,"threads_started":1}
W20260806 02:12:50.851462 18713 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/krb5cc' of type FILE
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:50.852532 18713 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:50.853427 18713 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.65@KRBTEST.COM
I20260806 02:12:50.862701 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:50.854223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34758 (local address 127.18.70.65:32993)
0806 02:12:50.854353 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:50.854357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:50.854441 (+ 84us) server_negotiation.cc:407] Connection header received
0806 02:12:50.854597 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:50.854600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:50.854644 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:50.854724 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:50.855264 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.856166 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.857012 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.857379 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.857491 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:50.859615 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:50.859623 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:50.859636 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:50.859656 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:50.861019 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.861445 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.861450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.861451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.861525 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.861787 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.861793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.861797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.862028 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:50.862233 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:50.862415 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 02:12:50.862586 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 02:12:50.866242 24986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:12:50.865405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34766 (local address 127.18.70.65:32993)
0806 02:12:50.865518 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:50.865523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:50.865628 (+ 105us) server_negotiation.cc:407] Connection header received
0806 02:12:50.865702 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:50.865704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:50.865735 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:50.865816 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:50.866145 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34766: BlockingRecv error: recv got EOF from 127.0.0.1:34766 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 02:12:50.866305 18713 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/krb5cc' of type FILE
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:50.867159 18713 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:50.868012 18713 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.65@KRBTEST.COM
I20260806 02:12:50.875869 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:50.868722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34776 (local address 127.18.70.65:32993)
0806 02:12:50.868835 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:50.868840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:50.868951 (+ 111us) server_negotiation.cc:407] Connection header received
0806 02:12:50.869066 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:50.869068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:50.869099 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:50.869157 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:50.869675 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.870272 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.871007 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.871230 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.871312 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:50.873288 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:50.873291 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:50.873293 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:50.873312 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:50.874475 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.874842 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.874845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.874846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.874912 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.875125 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.875127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.875128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.875265 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:50.875356 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:50.875533 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 02:12:50.875726 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 02:12:50.878109 24986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:12:50.877345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34784 (local address 127.18.70.65:32993)
0806 02:12:50.877463 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:50.877467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:50.877486 (+ 19us) server_negotiation.cc:407] Connection header received
0806 02:12:50.877582 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:50.877584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:50.877615 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:50.877675 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:50.878000 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34784: BlockingRecv error: recv got EOF from 127.0.0.1:34784 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 02:12:50.878177 18713 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785982339772911-18713-0/krb5kdc/krb5cc' of type FILE
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:50.879092 18713 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:50.879981 18713 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.65@KRBTEST.COM
I20260806 02:12:50.887528 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:50.880735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34800 (local address 127.18.70.65:32993)
0806 02:12:50.880855 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:50.880858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:50.880909 (+ 51us) server_negotiation.cc:407] Connection header received
0806 02:12:50.880991 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:50.880993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:50.881024 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:50.881085 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:50.881504 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.881964 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.882789 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:50.882983 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:50.883054 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:50.885026 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:50.885029 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:50.885030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:50.885051 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:50.886271 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.886605 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.886608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.886609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.886674 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:50.886882 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:50.886885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:50.886886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:50.887016 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:50.887105 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:50.887281 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 02:12:50.887387 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 02:12:50.888410 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 24566
I20260806 02:12:50.897092 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 24705
I20260806 02:12:50.903726 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 24847
I20260806 02:12:50.911514 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 24467
2026-08-06T02:12:50Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3395 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24999](info): set up 2 sockets
Aug 06 02:12:50 dist-test-slave-6k22 krb5kdc[24999](info): commencing operation
krb5kdc: starting...
W20260806 02:12:51.958045 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:12:52 dist-test-slave-6k22 krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:12:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.126.keytab.
Entry for principal oryx/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.126.keytab.
I20260806 02:12:52.082616 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:45685
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.126.keytab
--principal=oryx/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:45685
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:52.194561 25013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:52.194846 25013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:52.194928 25013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:52.198942 25013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:52.199014 25013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:52.199052 25013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:52.199105 25013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:52.199146 25013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:52.204170 25013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:45685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.126.keytab
--principal=oryx/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:45685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:52.205109 25013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:52.206054 25013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:52.214591 25020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:52.214633 25019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:52.214675 25013 server_base.cc:1034] running on GCE node
W20260806 02:12:52.214676 25022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:52.215411 25013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:52.216460 25013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:52.217660 25013 hybrid_clock.cc:648] HybridClock initialized: now 1785982372217617 us; error 40 us; skew 500 ppm
Aug 06 02:12:52 dist-test-slave-6k22 krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982372, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:52.220366 25013 init.cc:377] Logged in from keytab as oryx/127.18.70.126@KRBTEST.COM (short username oryx)
I20260806 02:12:52.221196 25013 webserver.cc:460] Webserver started at http://127.18.70.126:42725/ using document root <none> and password file <none>
I20260806 02:12:52.221720 25013 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:52.221804 25013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:52.222002 25013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:52.224009 25013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "65c662a093244164b5077e524674f6bd"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "2eaacb3444da139ac4c1b3c745dc6b1e"
server_key_iv: "7113e35048c394070340b59c0f69d0b3"
server_key_version: "encryptionkey@0"
I20260806 02:12:52.224581 25013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "65c662a093244164b5077e524674f6bd"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "2eaacb3444da139ac4c1b3c745dc6b1e"
server_key_iv: "7113e35048c394070340b59c0f69d0b3"
server_key_version: "encryptionkey@0"
I20260806 02:12:52.228169 25013 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 02:12:52.230424 25029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:52.231678 25013 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:52.231806 25013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "65c662a093244164b5077e524674f6bd"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "2eaacb3444da139ac4c1b3c745dc6b1e"
server_key_iv: "7113e35048c394070340b59c0f69d0b3"
server_key_version: "encryptionkey@0"
I20260806 02:12:52.231920 25013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:52.241308 25013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:52.243901 25013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:52.244077 25013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:52.250732 25013 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:45685
I20260806 02:12:52.251058 25085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:45685 every 8 connection(s)
I20260806 02:12:52.251765 25013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:52.254438 25086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:52.258256 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 25013
I20260806 02:12:52.258421 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:52.258718 18713 external_mini_cluster.cc:1428] Setting key 0480e11e6ef039b0eeeb99ed6ff64134
I20260806 02:12:52.260406 25086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd: Bootstrap starting.
I20260806 02:12:52.262797 25086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:52.263643 25086 log.cc:824] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:52.265717 25086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd: No bootstrap required, opened a new log
Aug 06 02:12:52 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
I20260806 02:12:52.269886 25086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } }
I20260806 02:12:52.270087 25086 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:52.270120 25086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65c662a093244164b5077e524674f6bd, State: Initialized, Role: FOLLOWER
I20260806 02:12:52.270601 25086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } }
I20260806 02:12:52.270722 25086 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:52.270802 25086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:52.270921 25086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:52.271795 25086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } }
I20260806 02:12:52.272122 25086 leader_election.cc:302] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65c662a093244164b5077e524674f6bd; no voters:
I20260806 02:12:52.272442 25086 leader_election.cc:288] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:52.272578 25091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:52.272796 25091 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 1 LEADER]: Becoming Leader. State: Replica: 65c662a093244164b5077e524674f6bd, State: Running, Role: LEADER
I20260806 02:12:52.273083 25091 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } }
I20260806 02:12:52.273725 25086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:52.274883 25092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } }
I20260806 02:12:52.275051 25092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:52.275454 25093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65c662a093244164b5077e524674f6bd. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } }
I20260806 02:12:52.275584 25093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:52.275519 25100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:52.275707 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:52.260208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.18.70.126:45685)
0806 02:12:52.260713 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:52.260735 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:52.260775 (+ 40us) server_negotiation.cc:407] Connection header received
0806 02:12:52.261480 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:52.261497 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:52.261867 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:52.262184 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:52.263141 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.264339 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.265523 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.266136 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.266272 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:52.269275 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:52.269281 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:52.269295 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:52.269325 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:52.271687 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:52.272249 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:52.272256 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:52.272263 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:52.272398 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:52.272811 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:52.272815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:52.272817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:52.273524 (+ 707us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:52.273866 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:52.274105 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 02:12:52.274462 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":153,"threads_started":1}
I20260806 02:12:52.279379 25100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:52.285534 25100 catalog_manager.cc:1347] Generated new cluster ID: 93d9857deab442bda1973b5747078e73
I20260806 02:12:52.285604 25100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:52.307042 25100 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:52.308348 25100 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:52.319281 25100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd: Generated new TSK 0
I20260806 02:12:52.320084 25100 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab.
Entry for principal oryx/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab.
I20260806 02:12:52.356964 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45685
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab
--principal=oryx/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:52.466199 25112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:52.466431 25112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:52.466470 25112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:52.470391 25112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:52.470436 25112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:52.470533 25112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:12:52.475548 25112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab
--principal=oryx/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:52.476559 25112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:52.477519 25112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:52.486938 25118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:52.486953 25119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:52.487042 25121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:52.487200 25112 server_base.cc:1034] running on GCE node
I20260806 02:12:52.487689 25112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:52.488318 25112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:52.489481 25112 hybrid_clock.cc:648] HybridClock initialized: now 1785982372489465 us; error 27 us; skew 500 ppm
Aug 06 02:12:52 dist-test-slave-6k22 krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982372, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:52.492204 25112 init.cc:377] Logged in from keytab as oryx/127.18.70.65@KRBTEST.COM (short username oryx)
I20260806 02:12:52.493119 25112 webserver.cc:460] Webserver started at http://127.18.70.65:36307/ using document root <none> and password file <none>
I20260806 02:12:52.493661 25112 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:52.493719 25112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:52.493943 25112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:52.495978 25112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "15eaf475d67a46b0bc51aa2995c6fca0"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "b2eec092897d4c16a7f119aca95a6d26"
server_key_iv: "88e80c2ceb32390a0411786a50c9daa2"
server_key_version: "encryptionkey@0"
I20260806 02:12:52.496529 25112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "15eaf475d67a46b0bc51aa2995c6fca0"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "b2eec092897d4c16a7f119aca95a6d26"
server_key_iv: "88e80c2ceb32390a0411786a50c9daa2"
server_key_version: "encryptionkey@0"
I20260806 02:12:52.500103 25112 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:12:52.502522 25128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:52.503777 25112 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:52.503887 25112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "15eaf475d67a46b0bc51aa2995c6fca0"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "b2eec092897d4c16a7f119aca95a6d26"
server_key_iv: "88e80c2ceb32390a0411786a50c9daa2"
server_key_version: "encryptionkey@0"
I20260806 02:12:52.503981 25112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:52.516876 25112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:52.520280 25112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:52.520478 25112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:52.521030 25112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:52.521800 25112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:52.521845 25112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:52.521905 25112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:52.521936 25112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:52.531148 25112 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:41273
I20260806 02:12:52.531179 25246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:41273 every 8 connection(s)
I20260806 02:12:52.532055 25112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:12:52.533252 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 25112
I20260806 02:12:52.533381 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:12:52.533654 18713 external_mini_cluster.cc:1428] Setting key 98c4eab8a357663c8ddb33868370470c
Aug 06 02:12:52 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982372, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.70.66@KRBTEST.COM; defaulting to no policy
I20260806 02:12:52.547426 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:52.534039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:59075 (local address 127.18.70.126:45685)
0806 02:12:52.534159 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:52.534169 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:52.535217 (+ 1048us) server_negotiation.cc:407] Connection header received
0806 02:12:52.537382 (+ 2165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:52.537388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:52.537456 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:52.537562 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:52.538738 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.539320 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.540491 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.540944 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.541073 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:52.543904 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:52.543908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:52.543910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:52.543934 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:52.545362 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:52.545948 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:52.545953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:52.545958 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:52.546048 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:52.546380 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:52.546382 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:52.546383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:52.546577 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:52.546652 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:52.547192 (+ 540us) server_negotiation.cc:299] Negotiation successful
0806 02:12:52.547302 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.70.66@KRBTEST.COM" created.
I20260806 02:12:52.548374 25249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:52.534510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.65:59075)
0806 02:12:52.535064 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:52.535119 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:52.537118 (+ 1999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:52.537748 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:52.537755 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:52.538093 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:52.538584 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:52.538597 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.539429 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.539432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:52.540365 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:52.540375 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.541118 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.541122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:52.541289 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:52.541917 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:52.541976 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:52.543730 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:52.545481 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:52.545488 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:52.545503 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:52.545809 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:52.546135 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:52.546141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:52.546144 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:52.546292 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:52.546671 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:52.546678 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:52.546981 (+ 303us) client_negotiation.cc:769] Sending connection context
0806 02:12:52.547130 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 02:12:52.547374 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":145,"threads_started":1}
I20260806 02:12:52.549644 25247 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45685
I20260806 02:12:52.549912 25247 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:52.550509 25247 heartbeater.cc:511] Master 127.18.70.126:45685 requested a full tablet report, sending...
I20260806 02:12:52.552150 25047 ts_manager.cc:194] Registered new tserver with Master: 15eaf475d67a46b0bc51aa2995c6fca0 (127.18.70.65:41273)
I20260806 02:12:52.553283 25047 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:59075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab.
Entry for principal oryx/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab.
I20260806 02:12:52.564783 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45685
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab
--principal=oryx/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:52.676036 25252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:52.676270 25252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:52.676309 25252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:52.680239 25252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:52.680282 25252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:52.680334 25252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:12:52.685274 25252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab
--principal=oryx/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:52.686158 25252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:52.687089 25252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:52.696343 25257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:52.696432 25260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:52.696480 25258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:52.696684 25252 server_base.cc:1034] running on GCE node
I20260806 02:12:52.697036 25252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:52.697628 25252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:52.698828 25252 hybrid_clock.cc:648] HybridClock initialized: now 1785982372698810 us; error 26 us; skew 500 ppm
Aug 06 02:12:52 dist-test-slave-6k22 krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982372, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:52.701550 25252 init.cc:377] Logged in from keytab as oryx/127.18.70.66@KRBTEST.COM (short username oryx)
I20260806 02:12:52.702406 25252 webserver.cc:460] Webserver started at http://127.18.70.66:43323/ using document root <none> and password file <none>
I20260806 02:12:52.702945 25252 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:52.703029 25252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:52.703258 25252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:52.705260 25252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "6bad949c005549699519cf75f71049d3"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "f0e2e426c41adaf59f967a3b82b79c44"
server_key_iv: "bb1b5b390d44782404a15cfd388bc5eb"
server_key_version: "encryptionkey@0"
I20260806 02:12:52.705817 25252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "6bad949c005549699519cf75f71049d3"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "f0e2e426c41adaf59f967a3b82b79c44"
server_key_iv: "bb1b5b390d44782404a15cfd388bc5eb"
server_key_version: "encryptionkey@0"
I20260806 02:12:52.709602 25252 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 02:12:52.712031 25267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:52.713168 25252 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:52.713295 25252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "6bad949c005549699519cf75f71049d3"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "f0e2e426c41adaf59f967a3b82b79c44"
server_key_iv: "bb1b5b390d44782404a15cfd388bc5eb"
server_key_version: "encryptionkey@0"
I20260806 02:12:52.713399 25252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:52.731472 25252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:52.734607 25252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:52.734826 25252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:52.735425 25252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:52.736274 25252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:52.736348 25252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:52.736407 25252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:52.736449 25252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:52.745258 25252 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:42507
I20260806 02:12:52.745316 25389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:42507 every 8 connection(s)
I20260806 02:12:52.746114 25252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:12:52.751000 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 25252
I20260806 02:12:52.751132 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:12:52.751472 18713 external_mini_cluster.cc:1428] Setting key dac8ce0cee30f0dfb5bc5011a89db66e
Aug 06 02:12:52 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982372, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
I20260806 02:12:52.760370 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:52.747931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:54333 (local address 127.18.70.126:45685)
0806 02:12:52.748079 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:52.748083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:52.748841 (+ 758us) server_negotiation.cc:407] Connection header received
0806 02:12:52.750127 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:52.750129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:52.750175 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:52.750254 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:52.751443 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.752195 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.753553 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.753933 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.754054 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:52.756379 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:52.756386 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:52.756391 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:52.756424 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:52.758085 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:52.758659 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:52.758663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:52.758665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:52.758760 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:52.759088 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:52.759094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:52.759097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:52.759353 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:52.759471 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:52.760005 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 02:12:52.760145 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 02:12:52.760946 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:52.748297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.66:54333)
0806 02:12:52.748699 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:52.748739 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:52.749943 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:52.750413 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:52.750419 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:52.750753 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:52.751297 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:52.751314 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.752337 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.752340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:52.753434 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:52.753444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.754067 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.754069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:52.754201 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:52.754748 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:52.754764 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:52.756217 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:52.758196 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:52.758201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:52.758211 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:52.758555 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:52.758866 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:52.758868 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:52.758869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:52.758994 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:52.759457 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:52.759462 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:52.759768 (+ 306us) client_negotiation.cc:769] Sending connection context
0806 02:12:52.759918 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 02:12:52.760140 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":103,"threads_started":1}
I20260806 02:12:52.762037 25390 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45685
I20260806 02:12:52.762291 25390 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.18.70.67@KRBTEST.COM; defaulting to no policy
I20260806 02:12:52.762881 25390 heartbeater.cc:511] Master 127.18.70.126:45685 requested a full tablet report, sending...
I20260806 02:12:52.763998 25047 ts_manager.cc:194] Registered new tserver with Master: 6bad949c005549699519cf75f71049d3 (127.18.70.66:42507)
I20260806 02:12:52.764525 25047 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:54333
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.70.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab.
Entry for principal oryx/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab.
I20260806 02:12:52.780210 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45685
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab
--principal=oryx/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:52.889802 25395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:52.890015 25395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:52.890054 25395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:52.894116 25395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:52.894162 25395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:52.894215 25395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:12:52.899204 25395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab
--principal=oryx/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:52.900169 25395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:52.901075 25395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:52.910012 25401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:52.909996 25400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:52.909987 25404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:52.910223 25395 server_base.cc:1034] running on GCE node
I20260806 02:12:52.910704 25395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:52.911320 25395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:52.912494 25395 hybrid_clock.cc:648] HybridClock initialized: now 1785982372912459 us; error 46 us; skew 500 ppm
Aug 06 02:12:52 dist-test-slave-6k22 krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982372, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:52.915098 25395 init.cc:377] Logged in from keytab as oryx/127.18.70.67@KRBTEST.COM (short username oryx)
I20260806 02:12:52.916044 25395 webserver.cc:460] Webserver started at http://127.18.70.67:43015/ using document root <none> and password file <none>
I20260806 02:12:52.916596 25395 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:52.916679 25395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:52.916903 25395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:52.918838 25395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "17aeb8b024d0402d8b2282ced5edb90c"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "79f814afbb4987eb8fc097e7f2d62db8"
server_key_iv: "8675549c0cf8e14c4c686045be04bd77"
server_key_version: "encryptionkey@0"
I20260806 02:12:52.919378 25395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "17aeb8b024d0402d8b2282ced5edb90c"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "79f814afbb4987eb8fc097e7f2d62db8"
server_key_iv: "8675549c0cf8e14c4c686045be04bd77"
server_key_version: "encryptionkey@0"
I20260806 02:12:52.922801 25395 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 02:12:52.925067 25411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:52.926249 25395 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:52.926370 25395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "17aeb8b024d0402d8b2282ced5edb90c"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "79f814afbb4987eb8fc097e7f2d62db8"
server_key_iv: "8675549c0cf8e14c4c686045be04bd77"
server_key_version: "encryptionkey@0"
I20260806 02:12:52.926469 25395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:52.942651 25395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:52.945706 25395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:52.945948 25395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:52.946484 25395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:52.947263 25395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:52.947329 25395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:52.947396 25395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:52.947436 25395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:52.956859 25395 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:34655
I20260806 02:12:52.956881 25528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:34655 every 8 connection(s)
I20260806 02:12:52.957675 25395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:12:52.966353 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 25395
I20260806 02:12:52.966480 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
Aug 06 02:12:52 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982372, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
I20260806 02:12:52.966769 18713 external_mini_cluster.cc:1428] Setting key 53d23e859163adc1a5eabdcdd8fc0792
I20260806 02:12:52.970916 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:52.959392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:49641 (local address 127.18.70.126:45685)
0806 02:12:52.959521 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:52.959525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:52.960352 (+ 827us) server_negotiation.cc:407] Connection header received
0806 02:12:52.961642 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:52.961645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:52.961705 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:52.961817 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:52.962904 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.963389 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.964576 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.964807 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.964915 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:52.967311 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:52.967315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:52.967318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:52.967346 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:52.968950 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:52.969463 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:52.969468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:52.969469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:52.969554 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:52.969876 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:52.969881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:52.969883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:52.970063 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:52.970165 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:52.970631 (+ 466us) server_negotiation.cc:299] Negotiation successful
0806 02:12:52.970742 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 02:12:52.971674 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:52.959772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.67:49641)
0806 02:12:52.960203 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:52.960243 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:52.961423 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:52.961978 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:52.961985 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:52.962320 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:52.962776 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:52.962789 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.963526 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.963530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:52.964454 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:52.964464 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:52.964956 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:52.964960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:52.965127 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:52.965619 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:52.965635 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:52.967147 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:52.969063 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:52.969069 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:52.969082 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:52.969361 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:52.969648 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:52.969650 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:52.969652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:52.969785 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:52.970165 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:52.970173 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:52.970415 (+ 242us) client_negotiation.cc:769] Sending connection context
0806 02:12:52.970590 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 02:12:52.970883 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":99,"threads_started":1}
I20260806 02:12:52.972679 25529 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45685
I20260806 02:12:52.972860 25529 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:52.973281 25529 heartbeater.cc:511] Master 127.18.70.126:45685 requested a full tablet report, sending...
I20260806 02:12:52.974267 25047 ts_manager.cc:194] Registered new tserver with Master: 17aeb8b024d0402d8b2282ced5edb90c (127.18.70.67:34655)
I20260806 02:12:52.974924 25047 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:49641
I20260806 02:12:52.980448 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.70.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.70.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.70.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.125.keytab.
Entry for principal oryx/127.18.70.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.125.keytab.
I20260806 02:12:53.006690 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.125:41375
--webserver_interface=127.18.70.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.125.keytab
--principal=oryx/127.18.70.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:45685,127.18.70.125:41375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:53.118670 25535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:53.118906 25535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:53.118942 25535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:53.122687 25535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:53.122740 25535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:53.122757 25535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:53.122773 25535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:53.122787 25535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:53.127836 25535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:45685,127.18.70.125:41375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.125.keytab
--principal=oryx/127.18.70.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.125:41375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.70.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
W20260806 02:12:53.127990 25535 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.70.126:45685,127.18.70.125:41375'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 02:12:53.128767 25535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:53.129743 25535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:53.138242 25543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:53.138458 25535 server_base.cc:1034] running on GCE node
W20260806 02:12:53.138293 25541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:53.138243 25540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:53.139079 25535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:53.140177 25535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:53.141376 25535 hybrid_clock.cc:648] HybridClock initialized: now 1785982373141363 us; error 25 us; skew 500 ppm
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:53.144017 25535 init.cc:377] Logged in from keytab as oryx/127.18.70.125@KRBTEST.COM (short username oryx)
I20260806 02:12:53.144888 25535 webserver.cc:460] Webserver started at http://127.18.70.125:44411/ using document root <none> and password file <none>
I20260806 02:12:53.145429 25535 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:53.145515 25535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:53.145737 25535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:53.147735 25535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/data/instance:
uuid: "a686923b459746f0acebe48c9100cc25"
format_stamp: "Formatted at 2026-08-06 02:12:53 on dist-test-slave-6k22"
server_key: "72ab9f1a9c54892ed93f74e1ff79f1f7"
server_key_iv: "6b1d8ddb16efa0dcd0701d2776ba4456"
server_key_version: "encryptionkey@0"
I20260806 02:12:53.148286 25535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/wal/instance:
uuid: "a686923b459746f0acebe48c9100cc25"
format_stamp: "Formatted at 2026-08-06 02:12:53 on dist-test-slave-6k22"
server_key: "72ab9f1a9c54892ed93f74e1ff79f1f7"
server_key_iv: "6b1d8ddb16efa0dcd0701d2776ba4456"
server_key_version: "encryptionkey@0"
I20260806 02:12:53.151856 25535 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:12:53.154167 25550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:53.155345 25535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:12:53.155469 25535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/wal
uuid: "a686923b459746f0acebe48c9100cc25"
format_stamp: "Formatted at 2026-08-06 02:12:53 on dist-test-slave-6k22"
server_key: "72ab9f1a9c54892ed93f74e1ff79f1f7"
server_key_iv: "6b1d8ddb16efa0dcd0701d2776ba4456"
server_key_version: "encryptionkey@0"
I20260806 02:12:53.155637 25535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:53.182278 25535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:53.185114 25535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:53.185266 25535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:53.192021 25535 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.125:41375
I20260806 02:12:53.192296 25604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.125:41375 every 8 connection(s)
I20260806 02:12:53.193029 25535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/data/info.pb
I20260806 02:12:53.195748 25605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:53.199383 25605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:12:53.202914 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 25535
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.125@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.70.125@KRBTEST.COM
I20260806 02:12:53.213012 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.201161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46062 (local address 127.18.70.126:45685)
0806 02:12:53.201288 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.201292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.202046 (+ 754us) server_negotiation.cc:407] Connection header received
0806 02:12:53.202907 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.202911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.202968 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.203062 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.204430 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.205085 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.206233 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.206459 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.206533 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.209121 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.209125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.209127 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.209157 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.210653 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.211281 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.211284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.211285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.211365 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.211760 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.211763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.211764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.211983 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.212058 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.212816 (+ 758us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.212904 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:12:53.213846 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.203813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37586 (local address 127.18.70.125:41375)
0806 02:12:53.204154 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.204165 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.204187 (+ 22us) server_negotiation.cc:407] Connection header received
0806 02:12:53.204279 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.204284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.204462 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.204618 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.205374 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.206139 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.207251 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.207655 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.207783 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.210190 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.210196 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.210208 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.210237 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.212019 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.212392 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.212415 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.212417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.212499 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.212764 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.212767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.212775 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.213011 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.213186 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.213509 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.213609 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":89,"threads_started":1}
I20260806 02:12:53.213850 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.201492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46062)
0806 02:12:53.201898 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.201938 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.202701 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.203291 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.203301 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.203771 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.204297 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.204310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.205235 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.205239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.206113 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.206123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.206595 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.206599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.206809 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.207349 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.207365 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.208986 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.210754 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.210782 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.210792 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.211170 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.211461 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.211465 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.211468 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.211658 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.212234 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.212238 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.212538 (+ 300us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.212734 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.212998 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"spinlock_wait_cycles":896,"thread_start_us":98,"threads_started":1}
I20260806 02:12:53.215672 25605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.125@KRBTEST.COM for oryx/127.18.70.125@KRBTEST.COM
I20260806 02:12:53.222515 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.216138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:41375 (local address 127.0.0.1:37600)
0806 02:12:53.216253 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.216277 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.216351 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.216537 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.216539 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.216708 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.216794 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.216799 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.217452 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.217456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.217984 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.217991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.218427 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.218430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.218547 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.218891 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.218901 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.220056 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.221372 (+ 1316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.221375 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.221376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.221551 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.221763 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.221765 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.221766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.221833 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.222135 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.222137 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.222211 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.222296 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.222381 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":18}
I20260806 02:12:53.222666 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.216186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37600 (local address 127.18.70.125:41375)
0806 02:12:53.216272 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.216277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.216317 (+ 40us) server_negotiation.cc:407] Connection header received
0806 02:12:53.216414 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.216419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.216453 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.216526 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.216892 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.217356 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.218084 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.218304 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.218429 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.220177 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.220180 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.220181 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.220201 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.221272 (+ 1071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.221625 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.221627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.221628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.221692 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.221900 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.221902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.221903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.222038 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.222130 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.222349 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.222451 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":35}
I20260806 02:12:53.225437 25605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: Bootstrap starting.
I20260806 02:12:53.227556 25605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:53.228325 25605 log.cc:824] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:53.230048 25605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: No bootstrap required, opened a new log
I20260806 02:12:53.232920 25605 raft_consensus.cc:348] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } }
I20260806 02:12:53.233090 25605 raft_consensus.cc:374] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:53.233167 25605 raft_consensus.cc:729] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a686923b459746f0acebe48c9100cc25, State: Initialized, Role: FOLLOWER
I20260806 02:12:53.233572 25605 consensus_queue.cc:260] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } }
I20260806 02:12:53.233983 25612 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } } }
I20260806 02:12:53.234149 25612 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:12:53.234679 25605 sys_catalog.cc:564] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 02:12:53.239461 25623 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:12:53.239543 25623 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:12:53.241762 25535 master_runner.cc:428] Detected that this master a686923b459746f0acebe48c9100cc25 is joining an existing cluster
I20260806 02:12:53.241827 25535 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.70.125:41375
I20260806 02:12:53.251737 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.243736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.18.70.126:45685)
0806 02:12:53.243869 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.243876 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.244213 (+ 337us) server_negotiation.cc:407] Connection header received
0806 02:12:53.244329 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.244331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.244368 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.244454 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.244976 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.245691 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.246618 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.246850 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.246957 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.248118 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.248124 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.248129 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.248156 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.249677 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.250349 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.250353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.250355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.250443 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.250841 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.250845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.250846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.251051 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.251116 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.251467 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.251597 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 02:12:53.252345 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.244092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37604 (local address 127.18.70.125:41375)
0806 02:12:53.244190 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.244197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.244495 (+ 298us) server_negotiation.cc:407] Connection header received
0806 02:12:53.244632 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.244635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.244675 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.244755 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.245268 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.245958 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.247093 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.247401 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.247482 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.249269 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.249273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.249276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.249299 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.250806 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.251205 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.251208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.251209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.251294 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.251505 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.251507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.251509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.251679 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.251737 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.251995 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.252208 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":36,"spinlock_wait_cycles":96128}
I20260806 02:12:53.252346 25629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.243736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:41375 (local address 127.0.0.1:37604)
0806 02:12:53.244430 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.244441 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.244551 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.244797 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.244800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.245035 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.245151 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.245158 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.246058 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.246061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.246962 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.246974 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.247685 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.247689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.247793 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.248247 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.248262 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.249132 (+ 870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.250923 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.250925 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.250927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.251119 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.251377 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.251379 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.251381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.251440 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.251785 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.251787 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.251850 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.252043 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.252160 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":107,"threads_started":1}
I20260806 02:12:53.252560 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.243699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46068)
0806 02:12:53.244131 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.244146 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.244242 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.244471 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.244474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.244681 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.244860 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.244868 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.245815 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.245818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.246474 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.246482 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.246970 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.246973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.247058 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.247587 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.247609 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.247945 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.250014 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.250017 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.250018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.250231 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.250630 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.250633 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.250635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.250723 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.251183 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.251185 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.251268 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.251948 (+ 680us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.252093 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":13,"spinlock_wait_cycles":17280,"thread_start_us":121,"threads_started":1}
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982372, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.126@KRBTEST.COM for oryx/127.18.70.125@KRBTEST.COM
I20260806 02:12:53.266582 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.256757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:41375 (local address 127.0.0.1:37606)
0806 02:12:53.257108 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.257157 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.257300 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.257794 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.257800 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.258187 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.258328 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.258336 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.259281 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.259285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.260314 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.260325 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.260816 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.260819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.260925 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.261545 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.261559 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.263220 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.264933 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.264942 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.264948 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.265227 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.265524 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.265526 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.265528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.265614 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.266025 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.266031 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.266143 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.266252 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.266440 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":101,"threads_started":1}
I20260806 02:12:53.267925 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.257439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37606 (local address 127.18.70.125:41375)
0806 02:12:53.257551 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.257555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.257566 (+ 11us) server_negotiation.cc:407] Connection header received
0806 02:12:53.257618 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.257620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.257659 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.257726 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.258456 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.259176 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.260432 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.260709 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.260802 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.263342 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.263346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.263348 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.263370 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.264822 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.265349 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.265352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.265354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.265436 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.265704 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.265707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.265709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.265923 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.267684 (+ 1761us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.267743 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.267841 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:12:53.269025 25047 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.70.125:41375: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a686923b459746f0acebe48c9100cc25"
member_type: NON_VOTER
last_known_addr {
host: "127.18.70.125"
port: 41375
}
attrs {
promote: true
}
}
dest_uuid: "65c662a093244164b5077e524674f6bd"
cas_config_opid_index: -1
I20260806 02:12:53.269795 25047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } }
I20260806 02:12:53.271477 25634 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a686923b459746f0acebe48c9100cc25 (127.18.70.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } } }
I20260806 02:12:53.272356 25579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:53.273463 25579 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 FOLLOWER]: Refusing update from remote peer 65c662a093244164b5077e524674f6bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 02:12:53.274071 25635 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:53.275486 25634 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.70.125:41375
I20260806 02:12:53.275985 25535 master.cc:537] Master@127.18.70.125:41375 shutting down...
WARNING: no policy specified for oryx/127.18.70.124@KRBTEST.COM; defaulting to no policy
I20260806 02:12:53.278497 25635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } } }
I20260806 02:12:53.278697 25635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.70.124@KRBTEST.COM" created.
I20260806 02:12:53.282303 25579 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a686923b459746f0acebe48c9100cc25 (127.18.70.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } } }
I20260806 02:12:53.283418 25638 mvcc.cc:205] Tried to move back new op lower bound from 7315383797015146496 to 7315383796831428608. Current Snapshot: MvccSnapshot[applied={T|T < 7315383796870938624}]
I20260806 02:12:53.285616 25612 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65c662a093244164b5077e524674f6bd. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } } }
I20260806 02:12:53.285719 25612 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: This master's current role is: LEARNER
I20260806 02:12:53.285818 25612 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } } }
I20260806 02:12:53.285888 25612 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: This master's current role is: LEARNER
I20260806 02:12:53.286059 25641 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } } }
I20260806 02:12:53.286139 25641 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: This master's current role is: LEARNER
I20260806 02:12:53.287531 25535 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 LEARNER]: Raft consensus shutting down.
I20260806 02:12:53.287827 25535 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 LEARNER]: Raft consensus is shut down!
I20260806 02:12:53.287926 25535 tablet_replica.cc:330] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: stopping tablet replica
I20260806 02:12:53.294456 25535 master.cc:552] Master@127.18.70.125:41375 shutdown complete.
I20260806 02:12:53.294533 25535 master_runner.cc:305] Clearing existing system tablet
I20260806 02:12:53.294803 25535 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:53.294869 25535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:53.296059 25535 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 02:12:53.296904 25647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:53.297374 25535 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:53.297513 25535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:53.297549 25535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/wal
uuid: "a686923b459746f0acebe48c9100cc25"
format_stamp: "Formatted at 2026-08-06 02:12:53 on dist-test-slave-6k22"
server_key: "72ab9f1a9c54892ed93f74e1ff79f1f7"
server_key_iv: "6b1d8ddb16efa0dcd0701d2776ba4456"
server_key_version: "encryptionkey@0"
I20260806 02:12:53.299326 25535 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.70.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.124.keytab.
Entry for principal oryx/127.18.70.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.124.keytab.
I20260806 02:12:53.302016 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.124:45749
--webserver_interface=127.18.70.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.124.keytab
--principal=oryx/127.18.70.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:45685,127.18.70.125:41375,127.18.70.124:45749
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:12:53.302035 25535 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 02:12:53.302126 25535 log.cc:1196] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 02:12:53.302430 25535 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: Deleting consensus metadata
I20260806 02:12:53.302597 25535 master_runner.cc:315] Copying system tablet from 127.18.70.126:45685
I20260806 02:12:53.303903 25535 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: tablet copy: Beginning tablet copy session from remote peer at address 127.18.70.126:45685
I20260806 02:12:53.312995 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.304462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46084)
0806 02:12:53.304809 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.304824 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.304941 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.305195 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.305198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.305412 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.305520 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.305526 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.306225 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.306229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.307124 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.307135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.307854 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.307858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.307969 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.308545 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.308559 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.308961 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.311178 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.311182 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.311184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.311452 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.311796 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.311799 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.311802 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.311894 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.312454 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.312457 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.312550 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.312652 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.312802 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":99,"threads_started":1}
I20260806 02:12:53.313426 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.304546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46084 (local address 127.18.70.126:45685)
0806 02:12:53.304669 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.304673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.304919 (+ 246us) server_negotiation.cc:407] Connection header received
0806 02:12:53.305027 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.305030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.305090 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.305172 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.305627 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.306128 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.307245 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.307734 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.307857 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.309108 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.309114 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.309120 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.309151 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.311038 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.311575 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.311581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.311586 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.311683 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.312004 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.312009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.312011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.312334 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.312463 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.312754 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.312890 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 02:12:53.314249 25068 tablet_copy_service.cc:140] P 65c662a093244164b5077e524674f6bd: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a686923b459746f0acebe48c9100cc25 ({username='oryx', principal='oryx/127.18.70.125@KRBTEST.COM'} at 127.0.0.1:46084)
I20260806 02:12:53.314472 25068 tablet_copy_service.cc:161] P 65c662a093244164b5077e524674f6bd: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a686923b459746f0acebe48c9100cc25 at {username='oryx', principal='oryx/127.18.70.125@KRBTEST.COM'} at 127.0.0.1:46084: session id = a686923b459746f0acebe48c9100cc25-00000000000000000000000000000000
I20260806 02:12:53.316923 25068 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 02:12:53.318737 25535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 02:12:53.320896 25535 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: tablet copy: Starting download of 0 data blocks...
I20260806 02:12:53.321120 25535 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: tablet copy: Starting download of 1 WAL segments...
I20260806 02:12:53.323082 25535 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 02:12:53.325210 25068 tablet_copy_service.cc:342] P 65c662a093244164b5077e524674f6bd: Request end of tablet copy session a686923b459746f0acebe48c9100cc25-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.70.125@KRBTEST.COM'} at 127.0.0.1:46084
I20260806 02:12:53.325321 25068 tablet_copy_service.cc:433] P 65c662a093244164b5077e524674f6bd: ending tablet copy session a686923b459746f0acebe48c9100cc25-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a686923b459746f0acebe48c9100cc25
I20260806 02:12:53.332912 25535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:53.334483 25655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:53.334823 25658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:53.335175 25535 server_base.cc:1034] running on GCE node
W20260806 02:12:53.335372 25656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:53.335618 25535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:53.335968 25535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:53.337121 25535 hybrid_clock.cc:648] HybridClock initialized: now 1785982373337093 us; error 43 us; skew 500 ppm
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:53.338696 25535 init.cc:377] Logged in from keytab as oryx/127.18.70.125@KRBTEST.COM (short username oryx)
I20260806 02:12:53.339131 25535 webserver.cc:460] Webserver started at http://127.18.70.125:44651/ using document root <none> and password file <none>
I20260806 02:12:53.339319 25535 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:53.339365 25535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:53.340673 25535 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:53.341596 25665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:53.341820 25535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:53.341883 25535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/wal
uuid: "a686923b459746f0acebe48c9100cc25"
format_stamp: "Formatted at 2026-08-06 02:12:53 on dist-test-slave-6k22"
server_key: "72ab9f1a9c54892ed93f74e1ff79f1f7"
server_key_iv: "6b1d8ddb16efa0dcd0701d2776ba4456"
server_key_version: "encryptionkey@0"
I20260806 02:12:53.341974 25535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:53.366226 25535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:53.368355 25535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:53.373219 25535 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.125:41375
I20260806 02:12:53.373318 25722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.125:41375 every 8 connection(s)
I20260806 02:12:53.373826 25535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-1/data/info.pb
I20260806 02:12:53.375923 25723 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:12:53.376974 25723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: Bootstrap starting.
I20260806 02:12:53.385075 25723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:12:53.385416 25723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: Bootstrap complete.
I20260806 02:12:53.385995 25723 raft_consensus.cc:348] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } }
I20260806 02:12:53.386129 25723 raft_consensus.cc:729] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a686923b459746f0acebe48c9100cc25, State: Initialized, Role: LEARNER
I20260806 02:12:53.386279 25723 consensus_queue.cc:260] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } }
I20260806 02:12:53.386665 25727 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } } }
I20260806 02:12:53.386786 25723 sys_catalog.cc:564] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:53.386849 25727 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: This master's current role is: LEARNER
I20260806 02:12:53.389935 25738 catalog_manager.cc:1259] Loaded cluster ID: 93d9857deab442bda1973b5747078e73
I20260806 02:12:53.390138 25738 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: loading cluster ID for follower catalog manager: success
I20260806 02:12:53.392329 25738 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: acquiring CA information for follower catalog manager: success
I20260806 02:12:53.393790 25738 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 02:12:53.440424 25652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:53.440677 25652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:53.440745 25652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:53.444818 25652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:53.444875 25652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:53.444893 25652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:53.444929 25652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:53.444959 25652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:53.449993 25652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:45685,127.18.70.125:41375,127.18.70.124:45749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.124.keytab
--principal=oryx/127.18.70.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.124:45749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.70.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:53.451066 25652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:53.452085 25652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:53.460304 25747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:53.460316 25744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:53.460520 25652 server_base.cc:1034] running on GCE node
W20260806 02:12:53.460316 25745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:53.461197 25652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:53.462278 25652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:53.463454 25652 hybrid_clock.cc:648] HybridClock initialized: now 1785982373463427 us; error 46 us; skew 500 ppm
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:53.466239 25652 init.cc:377] Logged in from keytab as oryx/127.18.70.124@KRBTEST.COM (short username oryx)
I20260806 02:12:53.467095 25652 webserver.cc:460] Webserver started at http://127.18.70.124:41261/ using document root <none> and password file <none>
I20260806 02:12:53.467715 25652 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:53.467818 25652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:53.468047 25652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:53.470112 25652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/data/instance:
uuid: "a71d40aa02ae4b23980e6b02c76ab24a"
format_stamp: "Formatted at 2026-08-06 02:12:53 on dist-test-slave-6k22"
server_key: "61cf2547b35c0f23d9e45db1fbab067f"
server_key_iv: "b4279a8e7deaf6d398018c08b483326c"
server_key_version: "encryptionkey@0"
I20260806 02:12:53.470695 25652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/wal/instance:
uuid: "a71d40aa02ae4b23980e6b02c76ab24a"
format_stamp: "Formatted at 2026-08-06 02:12:53 on dist-test-slave-6k22"
server_key: "61cf2547b35c0f23d9e45db1fbab067f"
server_key_iv: "b4279a8e7deaf6d398018c08b483326c"
server_key_version: "encryptionkey@0"
I20260806 02:12:53.474377 25652 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 02:12:53.476868 25754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:53.478053 25652 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:53.478186 25652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/wal
uuid: "a71d40aa02ae4b23980e6b02c76ab24a"
format_stamp: "Formatted at 2026-08-06 02:12:53 on dist-test-slave-6k22"
server_key: "61cf2547b35c0f23d9e45db1fbab067f"
server_key_iv: "b4279a8e7deaf6d398018c08b483326c"
server_key_version: "encryptionkey@0"
I20260806 02:12:53.478293 25652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:53.489435 25652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:53.492594 25652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:53.492785 25652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:53.499750 25652 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.124:45749
I20260806 02:12:53.500011 25807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.124:45749 every 8 connection(s)
I20260806 02:12:53.500701 25652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/data/info.pb
I20260806 02:12:53.501812 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 25652
I20260806 02:12:53.504405 25808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:53.508762 25808 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.70.124@KRBTEST.COM
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.124@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
I20260806 02:12:53.516954 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.503141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57912 (local address 127.18.70.124:45749)
0806 02:12:53.503552 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.503605 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.503648 (+ 43us) server_negotiation.cc:407] Connection header received
0806 02:12:53.504298 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.504314 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.504676 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.504952 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.505857 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.506906 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.508139 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.508591 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.508708 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.511699 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.511703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.511735 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.511768 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.513795 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.514200 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.514207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.514212 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.514365 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.514648 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.514652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.514654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.515102 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.515312 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.515641 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.515951 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":110,"threads_started":1}
I20260806 02:12:53.520569 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.510922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46100 (local address 127.18.70.126:45685)
0806 02:12:53.511068 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.511073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.511380 (+ 307us) server_negotiation.cc:407] Connection header received
0806 02:12:53.511600 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.511603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.511656 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.511739 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.512583 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.513137 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.514298 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.514505 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.514612 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.517122 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.517126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.517129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.517153 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.518668 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.519270 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.519274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.519276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.519357 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.519681 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.519684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.519686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.519859 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.519964 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.520288 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.520400 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 02:12:53.520605 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.510821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46100)
0806 02:12:53.511227 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.511297 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.511427 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.511787 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.511794 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.512269 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.512447 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.512459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.513274 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.513278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.514194 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.514202 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.514630 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.514634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.514754 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.515257 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.515271 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.516942 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.518924 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.518932 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.518938 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.519155 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.519446 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.519449 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.519451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.519537 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.519965 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.519972 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.520084 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.520223 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.520401 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":130,"threads_started":1}
I20260806 02:12:53.521593 25808 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.124@KRBTEST.COM for oryx/127.18.70.125@KRBTEST.COM
I20260806 02:12:53.529186 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.522034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:41375 (local address 127.0.0.1:37618)
0806 02:12:53.522134 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.522145 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.522202 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.522603 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.522606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.522750 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.522843 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.522848 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.523791 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.523799 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.524294 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.524300 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.524745 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.524748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.524852 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.525193 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.525204 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.526253 (+ 1049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.527780 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.527783 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.527785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.527963 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.528237 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.528240 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.528242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.528319 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.528691 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.528694 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.528791 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.528888 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.529036 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 02:12:53.529238 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.522155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37618 (local address 127.18.70.125:41375)
0806 02:12:53.522408 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.522411 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.522423 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:12:53.522458 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.522460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.522497 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.522590 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.522964 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.523683 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.524407 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.524638 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.524745 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.526388 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.526391 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.526393 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.526414 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.527662 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.528068 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.528072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.528074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.528148 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.528430 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.528433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.528435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.528599 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.528698 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.528956 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.529072 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260806 02:12:53.529955 25808 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.124@KRBTEST.COM for oryx/127.18.70.124@KRBTEST.COM
I20260806 02:12:53.537501 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.530387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:45749 (local address 127.0.0.1:57926)
0806 02:12:53.530491 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.530506 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.530575 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.530722 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.530724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.530844 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.530945 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.530952 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.532065 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.532068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.532532 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.532539 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.532951 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.532954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.533019 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.533416 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.533429 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.534572 (+ 1143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.536348 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.536350 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.536352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.536552 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.536794 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.536797 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.536798 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.536867 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.537161 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.537163 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.537230 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.537315 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.537398 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 02:12:53.537613 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.530472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57926 (local address 127.18.70.124:45749)
0806 02:12:53.530569 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.530573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.530590 (+ 17us) server_negotiation.cc:407] Connection header received
0806 02:12:53.530622 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.530624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.530653 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.530708 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.531042 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.531943 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.532638 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.532852 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.532970 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.534714 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.534716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.534718 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.534764 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.536261 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.536636 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.536638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.536640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.536709 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.536938 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.536940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.536941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.537087 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.537159 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.537369 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.537452 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 02:12:53.540207 25808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: Bootstrap starting.
I20260806 02:12:53.542479 25808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:53.543196 25808 log.cc:824] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:53.545028 25808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: No bootstrap required, opened a new log
I20260806 02:12:53.547963 25808 raft_consensus.cc:348] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } }
I20260806 02:12:53.548144 25808 raft_consensus.cc:374] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:53.548219 25808 raft_consensus.cc:729] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a71d40aa02ae4b23980e6b02c76ab24a, State: Initialized, Role: FOLLOWER
I20260806 02:12:53.548627 25808 consensus_queue.cc:260] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } }
I20260806 02:12:53.549081 25815 sys_catalog.cc:455] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } } }
I20260806 02:12:53.549206 25815 sys_catalog.cc:458] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:12:53.549782 25808 sys_catalog.cc:564] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: configured and running, proceeding with master startup.
W20260806 02:12:53.554330 25827 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:12:53.554421 25827 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:12:53.556125 25247 heartbeater.cc:503] Master 127.18.70.126:45685 was elected leader, sending a full tablet report...
I20260806 02:12:53.557986 25652 master_runner.cc:428] Detected that this master a71d40aa02ae4b23980e6b02c76ab24a is joining an existing cluster
I20260806 02:12:53.558030 25652 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.70.124:45749
I20260806 02:12:53.568785 25834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.559964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46116)
0806 02:12:53.560237 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.560251 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.560348 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.560831 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.560834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.561061 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.561212 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.561220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.562342 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.562348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.563547 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.563578 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.564333 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.564337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.564594 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.565209 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.565220 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.565566 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.567418 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.567420 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.567422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.567675 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.567950 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.567952 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.567953 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.568025 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.568406 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.568408 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.568478 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.568553 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.568648 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":199,"thread_start_us":95,"threads_started":1}
I20260806 02:12:53.568892 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.560016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46116 (local address 127.18.70.126:45685)
0806 02:12:53.560626 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.560632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.560646 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:12:53.560684 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.560687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.560730 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.560823 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.561367 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.562186 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.563707 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.564067 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.564156 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.565701 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.565705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.565708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.565736 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.567314 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.567768 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.567772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.567773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.567866 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.568106 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.568109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.568111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.568321 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.568420 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.568628 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.568737 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527}
I20260806 02:12:53.570423 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 25112
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982372, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.126@KRBTEST.COM for oryx/127.18.70.124@KRBTEST.COM
I20260806 02:12:53.587864 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:41273
--local_ip_for_outbound_sockets=127.18.70.65
--tserver_master_addrs=127.18.70.126:45685
--webserver_port=36307
--webserver_interface=127.18.70.65
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab
--principal=oryx/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.70.126:45685,127.18.70.125:41375
--tserver_master_addrs=127.18.70.126:45685,127.18.70.125:41375,127.18.70.124:45749 with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:12:53.590222 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.560021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37624 (local address 127.18.70.125:41375)
0806 02:12:53.560132 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.560138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.560675 (+ 537us) server_negotiation.cc:407] Connection header received
0806 02:12:53.560794 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.560799 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.560847 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.560918 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.561477 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.562239 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.563707 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.564089 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.564165 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.585932 (+ 21767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.585937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.585941 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.585979 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.587801 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.588630 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.588633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.588635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.588765 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.589155 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.589159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.589163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.589432 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.589514 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.589910 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.590059 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 02:12:53.590947 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.561012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57942 (local address 127.18.70.124:45749)
0806 02:12:53.561106 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.561110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.561228 (+ 118us) server_negotiation.cc:407] Connection header received
0806 02:12:53.561375 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.561378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.561410 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.561452 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.562025 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.562783 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.563999 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.564321 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.564389 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.585599 (+ 21210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.585604 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.585608 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.585635 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.587814 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.588419 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.588423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.588428 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.588503 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.590009 (+ 1506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.590025 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.590027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.590226 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.590303 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.590673 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.590787 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 02:12:53.591473 25835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.560381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:45749 (local address 127.0.0.1:57942)
0806 02:12:53.561158 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.561170 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.561286 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.561562 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.561565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.561786 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.561907 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.561914 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.562926 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.562930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.563847 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.563858 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.564531 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.564535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.564651 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.584604 (+ 19953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.584755 (+ 151us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.585167 (+ 412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.587916 (+ 2749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.587920 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.587922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.588159 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.588907 (+ 748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.588910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.588912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.589027 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.590449 (+ 1422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.590454 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.590538 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.591118 (+ 580us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.591265 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"mutex_wait_us":32,"spinlock_wait_cycles":5632,"thread_start_us":74,"threads_started":1}
I20260806 02:12:53.591746 25836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.560226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:41375 (local address 127.0.0.1:37624)
0806 02:12:53.560590 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.560600 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.560704 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.561035 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.561040 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.561223 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.561360 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.561367 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.562356 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.562360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.563583 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.563589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.564333 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.564337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.564597 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.584014 (+ 19417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.584035 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.585419 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.588279 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.588286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.588291 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.588519 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.588907 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.588910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.588912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.589046 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.589591 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.589594 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.589713 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.591223 (+ 1510us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.591337 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":262,"spinlock_wait_cycles":17408,"thread_start_us":94,"threads_started":1}
I20260806 02:12:53.594453 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.577511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:45749 (local address 127.0.0.1:57958)
0806 02:12:53.577728 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.577745 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.577833 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.580067 (+ 2234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.580070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.580289 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.580386 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.580392 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.581173 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.581179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.581733 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.581741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.582395 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.582399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.582511 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.582990 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.583003 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.584966 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.592777 (+ 7811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.592780 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.592782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.593034 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.593458 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.593461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.593463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.593549 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.593924 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.593927 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.594016 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.594151 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.594302 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260806 02:12:53.594465 25837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.577565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57958 (local address 127.18.70.124:45749)
0806 02:12:53.579796 (+ 2231us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.579803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.579817 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:12:53.579885 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.579887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.579930 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.580046 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.580511 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.581060 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.581860 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.582267 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.582393 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.585144 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.585148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.585152 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.585183 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.589904 (+ 4721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.593181 (+ 3277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.593184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.593186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.593282 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.593644 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.593649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.593650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.593831 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.593931 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.594209 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.594312 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2164,"thread_start_us":199,"threads_started":1}
I20260806 02:12:53.595357 25047 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.70.124:45749: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a"
member_type: NON_VOTER
last_known_addr {
host: "127.18.70.124"
port: 45749
}
attrs {
promote: true
}
}
dest_uuid: "65c662a093244164b5077e524674f6bd"
cas_config_opid_index: 5
I20260806 02:12:53.596014 25047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } }
I20260806 02:12:53.597057 25635 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a71d40aa02ae4b23980e6b02c76ab24a (127.18.70.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } } }
I20260806 02:12:53.597690 25781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:53.598308 25635 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.70.124:45749
I20260806 02:12:53.598531 25635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } } }
I20260806 02:12:53.598613 25635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:53.599380 25652 master.cc:537] Master@127.18.70.124:45749 shutting down...
I20260806 02:12:53.600085 25781 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 1 FOLLOWER]: Refusing update from remote peer 65c662a093244164b5077e524674f6bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 02:12:53.600792 25636 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 02:12:53.602265 25032 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 02:12:53.603860 25032 consensus_peers.cc:487] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd -> Peer a71d40aa02ae4b23980e6b02c76ab24a (127.18.70.124:45749): Couldn't send request to peer a71d40aa02ae4b23980e6b02c76ab24a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 02:12:53.604951 25652 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:53.605245 25652 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:53.605365 25652 tablet_replica.cc:330] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: stopping tablet replica
I20260806 02:12:53.607880 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.598933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:41375 (local address 127.0.0.1:37636)
0806 02:12:53.603247 (+ 4314us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.603263 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.603337 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.604115 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.604117 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.604134 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:53.604203 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.604207 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.605303 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.605306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.606473 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.606484 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.607436 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.607439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.607522 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.607596 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.607673 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.607763 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609}
I20260806 02:12:53.608003 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.598965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.18.70.125:41375)
0806 02:12:53.599094 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.599100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.603420 (+ 4320us) server_negotiation.cc:407] Connection header received
0806 02:12:53.603469 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.603474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.603537 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.603654 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:53.604316 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.605103 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.606674 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.607314 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.607437 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.607787 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.607824 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.607893 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 02:12:53.608646 25693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 LEARNER]: Refusing update from remote peer 65c662a093244164b5077e524674f6bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 02:12:53.610034 25636 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 02:12:53.611212 25693 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a71d40aa02ae4b23980e6b02c76ab24a (127.18.70.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } } }
I20260806 02:12:53.612042 25652 master.cc:552] Master@127.18.70.124:45749 shutdown complete.
I20260806 02:12:53.612123 25652 master_runner.cc:305] Clearing existing system tablet
I20260806 02:12:53.612402 25652 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:53.612495 25652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:53.613513 25727 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65c662a093244164b5077e524674f6bd. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } } }
I20260806 02:12:53.613639 25727 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: This master's current role is: LEARNER
I20260806 02:12:53.613770 25727 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: NON_VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: true } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } } }
I20260806 02:12:53.613862 25727 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: This master's current role is: LEARNER
I20260806 02:12:53.614089 25652 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:53.614889 25634 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd: attempting to promote NON_VOTER a686923b459746f0acebe48c9100cc25 to VOTER
I20260806 02:12:53.615289 25634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } }
I20260806 02:12:53.616627 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.616346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:45749 (local address 127.0.0.1:57962)
0806 02:12:53.616482 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.616556 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:45749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 02:12:53.616945 25693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 LEARNER]: Refusing update from remote peer 65c662a093244164b5077e524674f6bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 02:12:53.617316 25636 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 02:12:53.617864 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.617624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:45749 (local address 127.0.0.1:57974)
0806 02:12:53.617760 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.617806 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:45749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260806 02:12:53.618040 25032 consensus_peers.cc:487] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd -> Peer a71d40aa02ae4b23980e6b02c76ab24a (127.18.70.124:45749): Couldn't send request to peer a71d40aa02ae4b23980e6b02c76ab24a. Status: Network error: Client connection negotiation failed: client connection to 127.18.70.124:45749: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 02:12:53.618271 25843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:53.618528 25652 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:53.618652 25652 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 02:12:53.618705 25652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/wal
uuid: "a71d40aa02ae4b23980e6b02c76ab24a"
format_stamp: "Formatted at 2026-08-06 02:12:53 on dist-test-slave-6k22"
server_key: "61cf2547b35c0f23d9e45db1fbab067f"
server_key_iv: "b4279a8e7deaf6d398018c08b483326c"
server_key_version: "encryptionkey@0"
I20260806 02:12:53.619061 25636 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a686923b459746f0acebe48c9100cc25 (127.18.70.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } } }
I20260806 02:12:53.620456 25636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } } }
I20260806 02:12:53.620550 25636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:53.620668 25636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } } }
I20260806 02:12:53.620738 25636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:53.620882 25652 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:53.621400 25693 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a686923b459746f0acebe48c9100cc25 (127.18.70.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } } }
I20260806 02:12:53.622766 25841 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } } }
I20260806 02:12:53.622879 25841 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:12:53.623960 25652 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 02:12:53.624063 25652 log.cc:1196] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 02:12:53.624318 25652 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: Deleting consensus metadata
I20260806 02:12:53.624485 25652 master_runner.cc:315] Copying system tablet from 127.18.70.126:45685
I20260806 02:12:53.625554 25652 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: tablet copy: Beginning tablet copy session from remote peer at address 127.18.70.126:45685
I20260806 02:12:53.633507 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.626112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46128 (local address 127.18.70.126:45685)
0806 02:12:53.626217 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.626221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.626440 (+ 219us) server_negotiation.cc:407] Connection header received
0806 02:12:53.626560 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.626563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.626627 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.626714 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.627143 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.627722 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.628584 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.628819 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.628923 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.630099 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.630103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.630105 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.630128 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.631861 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.632300 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.632303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.632305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.632384 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.632658 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.632670 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.632672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.632849 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.632954 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.633254 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.633359 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 02:12:53.633519 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.626068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46128)
0806 02:12:53.626354 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.626365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.626483 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.626733 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.626736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.626948 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.627031 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.627037 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.627859 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.627862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.628456 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.628465 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.628941 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.628945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.629043 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.629640 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.629659 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.629965 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.631971 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.631976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.631978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.632186 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.632476 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.632480 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.632482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.632568 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.632948 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.632951 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.633051 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.633158 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.633306 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":90,"threads_started":1}
I20260806 02:12:53.634308 25068 tablet_copy_service.cc:140] P 65c662a093244164b5077e524674f6bd: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a71d40aa02ae4b23980e6b02c76ab24a ({username='oryx', principal='oryx/127.18.70.124@KRBTEST.COM'} at 127.0.0.1:46128)
I20260806 02:12:53.634403 25068 tablet_copy_service.cc:161] P 65c662a093244164b5077e524674f6bd: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a71d40aa02ae4b23980e6b02c76ab24a at {username='oryx', principal='oryx/127.18.70.124@KRBTEST.COM'} at 127.0.0.1:46128: session id = a71d40aa02ae4b23980e6b02c76ab24a-00000000000000000000000000000000
I20260806 02:12:53.635376 25068 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 02:12:53.636579 25652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 02:12:53.638351 25652 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: tablet copy: Starting download of 0 data blocks...
I20260806 02:12:53.638563 25652 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: tablet copy: Starting download of 1 WAL segments...
I20260806 02:12:53.640024 25652 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 02:12:53.641570 25068 tablet_copy_service.cc:342] P 65c662a093244164b5077e524674f6bd: Request end of tablet copy session a71d40aa02ae4b23980e6b02c76ab24a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.70.124@KRBTEST.COM'} at 127.0.0.1:46128
I20260806 02:12:53.641661 25068 tablet_copy_service.cc:433] P 65c662a093244164b5077e524674f6bd: ending tablet copy session a71d40aa02ae4b23980e6b02c76ab24a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a71d40aa02ae4b23980e6b02c76ab24a
I20260806 02:12:53.648468 25652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:53.650068 25850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:53.650365 25854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:53.650416 25652 server_base.cc:1034] running on GCE node
W20260806 02:12:53.654145 25851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:53.654382 25652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:53.654726 25652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:53.655891 25652 hybrid_clock.cc:648] HybridClock initialized: now 1785982373655874 us; error 33 us; skew 500 ppm
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:53.657756 25652 init.cc:377] Logged in from keytab as oryx/127.18.70.124@KRBTEST.COM (short username oryx)
I20260806 02:12:53.658222 25652 webserver.cc:460] Webserver started at http://127.18.70.124:43373/ using document root <none> and password file <none>
I20260806 02:12:53.658421 25652 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:53.658473 25652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:53.659755 25652 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:53.660705 25861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:53.660938 25652 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:53.661026 25652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/wal
uuid: "a71d40aa02ae4b23980e6b02c76ab24a"
format_stamp: "Formatted at 2026-08-06 02:12:53 on dist-test-slave-6k22"
server_key: "61cf2547b35c0f23d9e45db1fbab067f"
server_key_iv: "b4279a8e7deaf6d398018c08b483326c"
server_key_version: "encryptionkey@0"
I20260806 02:12:53.661106 25652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:53.685631 25652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:53.687920 25652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:53.693104 25652 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.124:45749
I20260806 02:12:53.693112 25915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.124:45749 every 8 connection(s)
I20260806 02:12:53.693758 25652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/master-2/data/info.pb
I20260806 02:12:53.695348 25916 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:12:53.696276 25916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: Bootstrap starting.
I20260806 02:12:53.704903 25916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:12:53.705207 25916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: Bootstrap complete.
I20260806 02:12:53.705699 25916 raft_consensus.cc:348] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } }
I20260806 02:12:53.705833 25916 raft_consensus.cc:729] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a71d40aa02ae4b23980e6b02c76ab24a, State: Initialized, Role: LEARNER
I20260806 02:12:53.705951 25916 consensus_queue.cc:260] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } }
I20260806 02:12:53.706298 25919 sys_catalog.cc:455] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } } }
I20260806 02:12:53.706465 25916 sys_catalog.cc:564] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:53.706476 25919 sys_catalog.cc:458] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: This master's current role is: LEARNER
I20260806 02:12:53.709347 25931 catalog_manager.cc:1259] Loaded cluster ID: 93d9857deab442bda1973b5747078e73
I20260806 02:12:53.709460 25931 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: loading cluster ID for follower catalog manager: success
I20260806 02:12:53.711392 25931 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: acquiring CA information for follower catalog manager: success
I20260806 02:12:53.712478 25931 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 02:12:53.747340 25838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:53.747733 25838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:53.747805 25838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:53.753532 25838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:53.753603 25838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:53.753685 25838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:12:53.761329 25838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab
--principal=oryx/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:41273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=36307
--tserver_master_addrs=127.18.70.126:45685,127.18.70.125:41375,127.18.70.124:45749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:53.762663 25838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:53.764086 25838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 02:12:53.766611 25390 heartbeater.cc:503] Master 127.18.70.126:45685 was elected leader, sending a full tablet report...
W20260806 02:12:53.774935 25937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:53.774936 25936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:53.775193 25939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:53.775354 25838 server_base.cc:1034] running on GCE node
I20260806 02:12:53.775799 25838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:53.776516 25838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:53.777694 25838 hybrid_clock.cc:648] HybridClock initialized: now 1785982373777658 us; error 43 us; skew 500 ppm
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:53.780364 25838 init.cc:377] Logged in from keytab as oryx/127.18.70.65@KRBTEST.COM (short username oryx)
I20260806 02:12:53.781255 25838 webserver.cc:460] Webserver started at http://127.18.70.65:36307/ using document root <none> and password file <none>
I20260806 02:12:53.781821 25838 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:53.781913 25838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:53.785789 25838 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 02:12:53.788352 25946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:53.789583 25838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 02:12:53.789721 25838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "15eaf475d67a46b0bc51aa2995c6fca0"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "b2eec092897d4c16a7f119aca95a6d26"
server_key_iv: "88e80c2ceb32390a0411786a50c9daa2"
server_key_version: "encryptionkey@0"
I20260806 02:12:53.790244 25838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:53.800879 25838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:53.803476 25838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:53.803689 25838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:53.804247 25838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:53.805075 25838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:53.805143 25838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:53.805210 25838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:53.805254 25838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:53.814976 25838 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:41273
I20260806 02:12:53.815012 26066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:41273 every 8 connection(s)
I20260806 02:12:53.815984 25838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:12:53.816349 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 25838
I20260806 02:12:53.816573 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 25252
I20260806 02:12:53.826725 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:42507
--local_ip_for_outbound_sockets=127.18.70.66
--tserver_master_addrs=127.18.70.126:45685
--webserver_port=43323
--webserver_interface=127.18.70.66
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab
--principal=oryx/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.70.126:45685,127.18.70.125:41375
--tserver_master_addrs=127.18.70.126:45685,127.18.70.125:41375,127.18.70.124:45749 with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.124@KRBTEST.COM
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.125@KRBTEST.COM
I20260806 02:12:53.835114 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.817870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:48737 (local address 127.18.70.126:45685)
0806 02:12:53.818147 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.818153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.819740 (+ 1587us) server_negotiation.cc:407] Connection header received
0806 02:12:53.821484 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.821490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.821551 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.821630 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.823095 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.823980 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.826101 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.826471 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.826559 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.831184 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.831187 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.831190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.831220 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.833194 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.833716 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.833719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.833721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.833813 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.834138 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.834141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.834143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.834347 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.834420 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.834831 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.834942 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260806 02:12:53.835705 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.820235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:56427 (local address 127.18.70.124:45749)
0806 02:12:53.821714 (+ 1479us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.821727 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.821740 (+ 13us) server_negotiation.cc:407] Connection header received
0806 02:12:53.821782 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.821785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.821835 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.821923 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.823100 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.824151 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.826127 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.826511 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.826599 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.830344 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.830347 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.830389 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.830417 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.832312 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.832998 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.833004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.833006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.833098 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.833588 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.833592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.833593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.833826 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.833921 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.835349 (+ 1428us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.835472 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1381,"thread_start_us":105,"threads_started":1}
I20260806 02:12:53.836598 26073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.819878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:45749 (local address 127.18.70.65:56427)
0806 02:12:53.821380 (+ 1502us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.821392 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.821488 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.822049 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.822052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.822213 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.822929 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.822947 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.824289 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.824292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.825986 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.825997 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.826757 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.826761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.827079 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.828046 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.828057 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.830174 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.832449 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.832455 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.832471 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.832819 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.833208 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.833211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.833215 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.833454 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.833969 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.833978 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.834358 (+ 380us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.834559 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.834891 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1425,"mutex_wait_us":721,"spinlock_wait_cycles":9216,"thread_start_us":77,"threads_started":1}
I20260806 02:12:53.836598 26072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.818445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.65:48737)
0806 02:12:53.819589 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.819643 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.820915 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.821764 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.821769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.822072 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.822929 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.822948 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.824126 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.824132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.825968 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.825982 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.826757 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.826761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.827294 (+ 533us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.828116 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.828136 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.831009 (+ 2873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.833348 (+ 2339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.833350 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.833352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.833608 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.833933 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.833938 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.833940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.834024 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.834492 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.834495 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.834586 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.835406 (+ 820us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.835538 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"mutex_wait_us":449,"spinlock_wait_cycles":7808,"thread_start_us":736,"threads_started":1}
I20260806 02:12:53.838325 26068 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45685
I20260806 02:12:53.838346 26067 heartbeater.cc:347] Connected to a master server at 127.18.70.124:45749
I20260806 02:12:53.838609 26067 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:53.839102 26067 heartbeater.cc:511] Master 127.18.70.124:45749 requested a full tablet report, sending...
I20260806 02:12:53.839764 26068 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:53.840153 26068 heartbeater.cc:511] Master 127.18.70.126:45685 requested a full tablet report, sending...
I20260806 02:12:53.840927 25879 ts_manager.cc:194] Registered new tserver with Master: 15eaf475d67a46b0bc51aa2995c6fca0 (127.18.70.65:41273)
I20260806 02:12:53.840965 25047 ts_manager.cc:194] Re-registered known tserver with Master: 15eaf475d67a46b0bc51aa2995c6fca0 (127.18.70.65:41273)
I20260806 02:12:53.841168 26071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.818869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:41375 (local address 127.18.70.65:41051)
0806 02:12:53.820801 (+ 1932us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:53.820817 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:53.820924 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:53.821518 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:53.821531 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:53.822069 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:53.822927 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.822948 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.824092 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.824095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.825986 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:53.825997 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.826758 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.826761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:53.827076 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.827998 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:53.828025 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:53.836982 (+ 8957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:53.839191 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.839194 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.839196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.839435 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.839831 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:53.839835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:53.839839 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:53.839942 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:53.840675 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:53.840678 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:53.840763 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 02:12:53.840879 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 02:12:53.841019 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1869,"mutex_wait_us":540,"spinlock_wait_cycles":15872,"thread_start_us":148,"threads_started":1}
I20260806 02:12:53.841625 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:53.818965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:41051 (local address 127.18.70.125:41375)
0806 02:12:53.819093 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:53.819097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:53.820928 (+ 1831us) server_negotiation.cc:407] Connection header received
0806 02:12:53.821147 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:53.821153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:53.821222 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:53.821326 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:53.823100 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.823926 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.826127 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:53.826488 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:53.826578 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:53.837145 (+ 10567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:53.837152 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:53.837157 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:53.837195 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:53.839052 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.839543 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.839548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.839549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.839682 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:53.840041 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:53.840058 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:53.840063 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:53.840291 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:53.840392 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:53.841365 (+ 973us) server_negotiation.cc:299] Negotiation successful
0806 02:12:53.841483 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 02:12:53.841756 25047 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:48737
I20260806 02:12:53.842233 26069 heartbeater.cc:347] Connected to a master server at 127.18.70.125:41375
I20260806 02:12:53.842319 26069 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:53.842485 26069 heartbeater.cc:511] Master 127.18.70.125:41375 requested a full tablet report, sending...
I20260806 02:12:53.843600 25683 ts_manager.cc:194] Registered new tserver with Master: 15eaf475d67a46b0bc51aa2995c6fca0 (127.18.70.65:41273)
W20260806 02:12:53.953088 26075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:53.953315 26075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:53.953352 26075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:53.958585 26075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:53.958638 26075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:53.958695 26075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:12:53.964028 26075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab
--principal=oryx/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:42507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=43323
--tserver_master_addrs=127.18.70.126:45685,127.18.70.125:41375,127.18.70.124:45749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:53.964916 26075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:53.965811 26075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:53.975251 26081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:53.975237 26083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:53.975306 26080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:53.975618 26075 server_base.cc:1034] running on GCE node
I20260806 02:12:53.976045 26075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:53.976671 26075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:53.976909 25529 heartbeater.cc:503] Master 127.18.70.126:45685 was elected leader, sending a full tablet report...
I20260806 02:12:53.977883 26075 hybrid_clock.cc:648] HybridClock initialized: now 1785982373977853 us; error 43 us; skew 500 ppm
Aug 06 02:12:53 dist-test-slave-6k22 krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:53.980672 26075 init.cc:377] Logged in from keytab as oryx/127.18.70.66@KRBTEST.COM (short username oryx)
I20260806 02:12:53.981532 26075 webserver.cc:460] Webserver started at http://127.18.70.66:43323/ using document root <none> and password file <none>
I20260806 02:12:53.982084 26075 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:53.982172 26075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:53.986006 26075 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 02:12:53.988209 26090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:53.989516 26075 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 02:12:53.989629 26075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "6bad949c005549699519cf75f71049d3"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "f0e2e426c41adaf59f967a3b82b79c44"
server_key_iv: "bb1b5b390d44782404a15cfd388bc5eb"
server_key_version: "encryptionkey@0"
I20260806 02:12:53.990172 26075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:54.014741 26075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:54.017593 26075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:54.017808 26075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:54.018421 26075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:54.019255 26075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:54.019305 26075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:54.019379 26075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:54.019411 26075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:54.027993 26075 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:42507
I20260806 02:12:54.028059 26208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:42507 every 8 connection(s)
I20260806 02:12:54.029011 26075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:12:54.034885 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 26075
I20260806 02:12:54.035073 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 25395
Aug 06 02:12:54 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.124@KRBTEST.COM
Aug 06 02:12:54 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.125@KRBTEST.COM
I20260806 02:12:54.046564 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.032555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:58905 (local address 127.18.70.124:45749)
0806 02:12:54.032683 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.032687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.032802 (+ 115us) server_negotiation.cc:407] Connection header received
0806 02:12:54.033408 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.033410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.033485 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.033556 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:54.037234 (+ 3678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.037741 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.039027 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.039284 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.039392 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.042334 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:54.042339 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:54.042342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:54.042382 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:54.044164 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.044648 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.044652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.044654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.044754 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.045160 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.045163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.045165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.045380 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:54.045491 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.046234 (+ 743us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.046391 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 02:12:54.047047 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.031427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:36015 (local address 127.18.70.125:41375)
0806 02:12:54.031545 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.031591 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.032338 (+ 747us) server_negotiation.cc:407] Connection header received
0806 02:12:54.033130 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.033134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.033196 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.033284 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:54.034750 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.035531 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.039544 (+ 4013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.040064 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.040202 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.042680 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:54.042687 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:54.042694 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:54.042731 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:54.044653 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.045085 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.045089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.045091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.045176 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.045476 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.045479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.045481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.045667 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:54.045737 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.046445 (+ 708us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.046578 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:12:54.047447 26215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.032423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:45749 (local address 127.18.70.66:58905)
0806 02:12:54.032716 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:54.032728 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:54.032903 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:54.033614 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:54.033617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:54.034027 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:54.034612 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.034623 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.037899 (+ 3276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.037905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.038904 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.038914 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.039428 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.039433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.039650 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.040514 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:54.040538 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:54.042086 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:54.044292 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:54.044298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:54.044311 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:54.044548 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:54.044870 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:54.044876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:54.044881 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:54.045070 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:54.045787 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:54.045795 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:54.046013 (+ 218us) client_negotiation.cc:769] Sending connection context
0806 02:12:54.046179 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 02:12:54.046520 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":67,"spinlock_wait_cycles":8064,"thread_start_us":107,"threads_started":1}
I20260806 02:12:54.047791 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.031227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:41375 (local address 127.18.70.66:36015)
0806 02:12:54.032219 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:54.032232 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:54.032896 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:54.033466 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:54.033475 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:54.034048 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:54.034592 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.034612 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.037995 (+ 3383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.038002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.039363 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.039380 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.040185 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.040188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.040267 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.040708 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:54.040731 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:54.042518 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:54.044753 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:54.044757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:54.044759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:54.044991 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:54.045269 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:54.045275 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:54.045280 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:54.045374 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:54.045784 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:54.045793 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:54.046022 (+ 229us) client_negotiation.cc:769] Sending connection context
0806 02:12:54.046180 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 02:12:54.047439 (+ 1259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"mutex_wait_us":1072,"spinlock_wait_cycles":42752,"thread_start_us":188,"threads_started":1}
I20260806 02:12:54.048775 26209 heartbeater.cc:347] Connected to a master server at 127.18.70.124:45749
I20260806 02:12:54.048808 26211 heartbeater.cc:347] Connected to a master server at 127.18.70.125:41375
I20260806 02:12:54.049026 26211 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:54.049029 26209 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:54.049546 26211 heartbeater.cc:511] Master 127.18.70.125:41375 requested a full tablet report, sending...
I20260806 02:12:54.049549 26209 heartbeater.cc:511] Master 127.18.70.124:45749 requested a full tablet report, sending...
I20260806 02:12:54.050625 25683 ts_manager.cc:194] Registered new tserver with Master: 6bad949c005549699519cf75f71049d3 (127.18.70.66:42507)
I20260806 02:12:54.050669 25879 ts_manager.cc:194] Registered new tserver with Master: 6bad949c005549699519cf75f71049d3 (127.18.70.66:42507)
I20260806 02:12:54.052098 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:34655
--local_ip_for_outbound_sockets=127.18.70.67
--tserver_master_addrs=127.18.70.126:45685
--webserver_port=43015
--webserver_interface=127.18.70.67
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab
--principal=oryx/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.70.126:45685,127.18.70.125:41375
--tserver_master_addrs=127.18.70.126:45685,127.18.70.125:41375,127.18.70.124:45749 with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 02:12:54 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982373, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
I20260806 02:12:54.060124 26213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.031227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.66:54385)
0806 02:12:54.031794 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:54.031833 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:54.032896 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:54.033466 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:54.033475 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:54.034042 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:54.034592 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.034612 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.052463 (+ 17851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.052469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.053402 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.053417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.054021 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.054024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.054090 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.054568 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:54.054580 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:54.056453 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:54.058595 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:54.058598 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:54.058600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:54.058824 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:54.059136 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:54.059138 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:54.059140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:54.059223 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:54.059761 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:54.059763 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:54.059838 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 02:12:54.059907 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 02:12:54.060003 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":113,"spinlock_wait_cycles":41472,"thread_start_us":728,"threads_started":1}
I20260806 02:12:54.060380 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.030855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:54385 (local address 127.18.70.126:45685)
0806 02:12:54.030989 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.030995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.031932 (+ 937us) server_negotiation.cc:407] Connection header received
0806 02:12:54.033144 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.033147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.033205 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.033259 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:54.051709 (+ 18450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.052326 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.053524 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.053893 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.054013 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.056663 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:54.056666 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:54.056668 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:54.056694 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:54.058485 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.058937 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.058943 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.058945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.059043 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.059323 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.059329 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.059333 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.059658 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:54.059767 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.060013 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.060153 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 02:12:54.060962 26210 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45685
I20260806 02:12:54.061064 26210 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:54.061264 26210 heartbeater.cc:511] Master 127.18.70.126:45685 requested a full tablet report, sending...
I20260806 02:12:54.061990 25047 ts_manager.cc:194] Re-registered known tserver with Master: 6bad949c005549699519cf75f71049d3 (127.18.70.66:42507)
I20260806 02:12:54.062538 25047 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:54385
I20260806 02:12:54.104423 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.100853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57986 (local address 127.18.70.124:45749)
0806 02:12:54.100979 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.100983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.101000 (+ 17us) server_negotiation.cc:407] Connection header received
0806 02:12:54.101141 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.101143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.101197 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.101270 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:54.101540 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.102057 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.103423 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.103922 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.104005 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.104118 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.104255 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.104313 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:12:54.104463 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.100719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:45749 (local address 127.0.0.1:57986)
0806 02:12:54.100893 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:54.100911 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:54.101019 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:54.101305 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:54.101308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:54.101327 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:54.101442 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.101448 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.102154 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.102157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.103309 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.103321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.104036 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.104040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.104132 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.104158 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 02:12:54.104239 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 02:12:54.104324 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 02:12:54.105154 25885 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 02:12:54.105544 25919 sys_catalog.cc:455] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65c662a093244164b5077e524674f6bd. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } } }
I20260806 02:12:54.105700 25919 sys_catalog.cc:458] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: This master's current role is: LEARNER
W20260806 02:12:54.168687 26216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:54.168939 26216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:54.169006 26216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:54.172919 26216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:54.172987 26216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:54.173074 26216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:12:54.178974 26216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab
--principal=oryx/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:34655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=43015
--tserver_master_addrs=127.18.70.126:45685,127.18.70.125:41375,127.18.70.124:45749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:54.180229 26216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:54.181571 26216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:54.191432 26222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:54.191506 26224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:54.191542 26216 server_base.cc:1034] running on GCE node
W20260806 02:12:54.191432 26221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:54.192087 26216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:54.192670 26216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:54.193853 26216 hybrid_clock.cc:648] HybridClock initialized: now 1785982374193824 us; error 25 us; skew 500 ppm
Aug 06 02:12:54 dist-test-slave-6k22 krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982374, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:54.196753 26216 init.cc:377] Logged in from keytab as oryx/127.18.70.67@KRBTEST.COM (short username oryx)
I20260806 02:12:54.197506 26216 webserver.cc:460] Webserver started at http://127.18.70.67:43015/ using document root <none> and password file <none>
I20260806 02:12:54.198014 26216 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:54.198082 26216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:54.201820 26216 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:54.203773 26232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:54.204929 26216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:12:54.205044 26216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "17aeb8b024d0402d8b2282ced5edb90c"
format_stamp: "Formatted at 2026-08-06 02:12:52 on dist-test-slave-6k22"
server_key: "79f814afbb4987eb8fc097e7f2d62db8"
server_key_iv: "8675549c0cf8e14c4c686045be04bd77"
server_key_version: "encryptionkey@0"
I20260806 02:12:54.205549 26216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:54.220944 26216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:54.224017 26216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:54.224191 26216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:54.224712 26216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:12:54.225535 26216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:54.225581 26216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:54.225644 26216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:54.225675 26216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:54.234992 26216 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:34655
I20260806 02:12:54.235092 26356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:34655 every 8 connection(s)
I20260806 02:12:54.236049 26216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:12:54.238922 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 26216
Aug 06 02:12:54 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982374, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
Aug 06 02:12:54 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982374, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.125@KRBTEST.COM
Aug 06 02:12:54 dist-test-slave-6k22 krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982374, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.124@KRBTEST.COM
I20260806 02:12:54.254549 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.239702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:39551 (local address 127.18.70.126:45685)
0806 02:12:54.239841 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.239847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.239938 (+ 91us) server_negotiation.cc:407] Connection header received
0806 02:12:54.240707 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.240711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.240774 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.240851 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:54.243132 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.243953 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.245807 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.246148 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.246235 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.249570 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:54.249574 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:54.249576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:54.249601 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:54.251895 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.253073 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.253076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.253078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.253146 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.253477 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.253480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.253482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.253650 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:54.253709 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.254299 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.254404 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 02:12:54.256408 26363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.239400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.67:39551)
0806 02:12:54.239848 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:54.239863 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:54.240377 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:54.241120 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:54.241129 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:54.242006 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:54.243010 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.243016 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.244175 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.244180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.245679 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.245689 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.246410 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.246414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.246635 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.247347 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:54.247378 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:54.249401 (+ 2023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:54.252666 (+ 3265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:54.252671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:54.252684 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:54.252969 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:54.253227 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:54.253229 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:54.253231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:54.253384 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:54.253791 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:54.253797 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:54.254038 (+ 241us) client_negotiation.cc:769] Sending connection context
0806 02:12:54.254960 (+ 922us) client_negotiation.cc:240] Negotiation successful
0806 02:12:54.255262 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":286,"spinlock_wait_cycles":10368,"thread_start_us":81,"threads_started":1}
I20260806 02:12:54.256824 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.237826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:45357 (local address 127.18.70.124:45749)
0806 02:12:54.237992 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.237998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.239059 (+ 1061us) server_negotiation.cc:407] Connection header received
0806 02:12:54.240630 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.240634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.240703 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.240765 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:54.243063 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.244062 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.250047 (+ 5985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.250390 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.250479 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.252940 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:54.252947 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:54.252951 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:54.252979 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:54.254786 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.255342 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.255346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.255348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.255430 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.255842 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.255845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.255847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.256085 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:54.256186 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.256516 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.256654 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":66}
I20260806 02:12:54.257309 26370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.241078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.18.70.126:45685)
0806 02:12:54.241471 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.241476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.242960 (+ 1484us) server_negotiation.cc:407] Connection header received
0806 02:12:54.243100 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.243103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.243155 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.243239 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:54.244390 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.245242 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.246683 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.247059 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.247125 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.249450 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:54.249454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:54.249457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:54.249484 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:54.252574 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.254356 (+ 1782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.254361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.254366 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.254465 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.256120 (+ 1655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.256126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.256128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.256335 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:54.256444 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.256983 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.257126 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":79,"threads_started":1}
I20260806 02:12:54.258458 26358 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45685
I20260806 02:12:54.258665 26358 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:54.259163 26358 heartbeater.cc:511] Master 127.18.70.126:45685 requested a full tablet report, sending...
I20260806 02:12:54.260196 25047 ts_manager.cc:194] Re-registered known tserver with Master: 17aeb8b024d0402d8b2282ced5edb90c (127.18.70.67:34655)
I20260806 02:12:54.260855 25047 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:39551
I20260806 02:12:54.261591 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.237936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:57925 (local address 127.18.70.125:41375)
0806 02:12:54.238088 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.238099 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.239440 (+ 1341us) server_negotiation.cc:407] Connection header received
0806 02:12:54.240715 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.240719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.240780 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.240858 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:54.244241 (+ 3383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.245135 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.246564 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.246946 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.247066 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.250434 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:54.250440 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:54.250443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:54.250473 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:54.258859 (+ 8386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.259755 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.259759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.259762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.260036 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.260393 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.260398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.260401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.260634 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:54.260735 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.261307 (+ 572us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.261428 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 02:12:54.261981 26362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.238390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:41375 (local address 127.18.70.67:57925)
0806 02:12:54.239341 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:54.239355 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:54.240364 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:54.241121 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:54.241129 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:54.242019 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:54.242876 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.242898 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.245310 (+ 2412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.245317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.246400 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.246412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.247085 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.247091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.247203 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.248619 (+ 1416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:54.248636 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:54.250255 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:54.258992 (+ 8737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:54.258996 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:54.258998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:54.259293 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:54.260170 (+ 877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:54.260173 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:54.260175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:54.260272 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:54.260785 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:54.260788 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:54.260880 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 02:12:54.261002 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 02:12:54.261800 (+ 798us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"mutex_wait_us":560,"spinlock_wait_cycles":4224,"thread_start_us":160,"threads_started":1}
I20260806 02:12:54.262753 26361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.238390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:45749 (local address 127.18.70.67:45357)
0806 02:12:54.238903 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:54.238957 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:54.240364 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:54.241235 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:54.241240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:54.242040 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:54.242875 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.242900 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.247206 (+ 4306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.247213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.248404 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.248416 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.250507 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.250511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.250632 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.251182 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:54.251199 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:54.252776 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:54.254932 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:54.254935 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:54.254937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:54.255222 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:54.255545 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:54.255548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:54.255550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:54.255716 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:54.256248 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:54.256251 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:54.256331 (+ 80us) client_negotiation.cc:769] Sending connection context
0806 02:12:54.257261 (+ 930us) client_negotiation.cc:240] Negotiation successful
0806 02:12:54.257418 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":93,"spinlock_wait_cycles":5888,"thread_start_us":110,"threads_started":1}
I20260806 02:12:54.263231 26359 heartbeater.cc:347] Connected to a master server at 127.18.70.125:41375
I20260806 02:12:54.263304 26359 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:54.263484 26359 heartbeater.cc:511] Master 127.18.70.125:41375 requested a full tablet report, sending...
I20260806 02:12:54.264102 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.241343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57992 (local address 127.18.70.124:45749)
0806 02:12:54.241599 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.241605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.243373 (+ 1768us) server_negotiation.cc:407] Connection header received
0806 02:12:54.243546 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.243549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.243631 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.243700 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:54.244667 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.245444 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.246675 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.247383 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.247483 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.249545 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:54.249549 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:54.249551 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:54.249575 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:54.253117 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.254085 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.254089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.254092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.254192 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.255103 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.255108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.255109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.255354 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:54.255488 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.256516 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.263924 (+ 7408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7254,"server-negotiator.queue_time_us":173,"thread_start_us":82,"threads_started":1}
I20260806 02:12:54.264235 25683 ts_manager.cc:194] Registered new tserver with Master: 17aeb8b024d0402d8b2282ced5edb90c (127.18.70.67:34655)
I20260806 02:12:54.261093 25046 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:12:54.265065 26357 heartbeater.cc:347] Connected to a master server at 127.18.70.124:45749
I20260806 02:12:54.265156 26357 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:54.265290 26357 heartbeater.cc:511] Master 127.18.70.124:45749 requested a full tablet report, sending...
W20260806 02:12:54.265566 25046 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:54.266002 25879 ts_manager.cc:194] Registered new tserver with Master: 17aeb8b024d0402d8b2282ced5edb90c (127.18.70.67:34655)
I20260806 02:12:54.268373 25634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: NON_VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: true } } }
I20260806 02:12:54.268532 25634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:54.273095 26379 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd: attempting to promote NON_VOTER a71d40aa02ae4b23980e6b02c76ab24a to VOTER
I20260806 02:12:54.273213 26373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.242141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37638 (local address 127.18.70.125:41375)
0806 02:12:54.242718 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.242723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.250266 (+ 7543us) server_negotiation.cc:407] Connection header received
0806 02:12:54.250857 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.250861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.250918 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.251000 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:54.265480 (+ 14480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.266115 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.266942 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.267168 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.267268 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.269162 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:54.269166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:54.269169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:54.269195 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:54.270949 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.271465 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.271469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.271471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.271605 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.271937 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.271941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.271943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.272189 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:54.272337 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.272886 (+ 549us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.273015 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":81,"threads_started":1}
I20260806 02:12:54.273526 26379 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } }
I20260806 02:12:54.274454 26378 raft_consensus.cc:1013] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 1 LEADER]: attempt to promote peer a71d40aa02ae4b23980e6b02c76ab24a: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260806 02:12:54.274691 25885 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 1 LEARNER]: Refusing update from remote peer 65c662a093244164b5077e524674f6bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 02:12:54.274701 25693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 FOLLOWER]: Refusing update from remote peer 65c662a093244164b5077e524674f6bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 02:12:54.275172 25634 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 02:12:54.275342 26378 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 02:12:54.277477 26377 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a71d40aa02ae4b23980e6b02c76ab24a (127.18.70.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } } }
I20260806 02:12:54.277918 25885 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a71d40aa02ae4b23980e6b02c76ab24a (127.18.70.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } } }
I20260806 02:12:54.278887 26378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } } }
I20260806 02:12:54.279016 26378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:54.278908 26377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } } }
I20260806 02:12:54.279165 26377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:54.280076 25919 sys_catalog.cc:455] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } } }
I20260806 02:12:54.280206 25919 sys_catalog.cc:458] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:12:54.280658 25634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } } }
I20260806 02:12:54.280794 25634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:54.281205 25693 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a71d40aa02ae4b23980e6b02c76ab24a (127.18.70.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } } }
I20260806 02:12:54.284984 26387 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65c662a093244164b5077e524674f6bd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } } }
I20260806 02:12:54.285130 26387 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:12:54.290043 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.283719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:42507 (local address 127.0.0.1:39374)
0806 02:12:54.284048 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:54.284062 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:54.284138 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:54.285675 (+ 1537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:54.285678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:54.285697 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:54.285801 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.285808 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.287236 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.287239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.288482 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.288493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.289665 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.289669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.289762 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.289789 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 02:12:54.289836 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 02:12:54.289898 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":30}
I20260806 02:12:54.290680 26389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.284376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39374 (local address 127.18.70.66:42507)
0806 02:12:54.285055 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.285070 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.285096 (+ 26us) server_negotiation.cc:407] Connection header received
0806 02:12:54.285225 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.285233 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.285496 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.285711 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:54.285922 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.287113 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.288671 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.289528 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.290236 (+ 708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.290401 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.290453 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.290538 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":111,"threads_started":1}
I20260806 02:12:54.291638 26392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.285222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37368 (local address 127.18.70.65:41273)
0806 02:12:54.286657 (+ 1435us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.286667 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.286689 (+ 22us) server_negotiation.cc:407] Connection header received
0806 02:12:54.286765 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.286835 (+ 70us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.287001 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.287192 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:54.287447 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.288595 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.289996 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.290898 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.291001 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.291128 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.291363 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.291458 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1335,"thread_start_us":94,"threads_started":1}
I20260806 02:12:54.291682 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.284050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:34655 (local address 127.0.0.1:42976)
0806 02:12:54.284643 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:54.284660 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:54.284763 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:54.285949 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:54.285956 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:54.285972 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:54.286079 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.286085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.288213 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.288219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.289180 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.289190 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.291041 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.291045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.291218 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.291236 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 02:12:54.291310 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 02:12:54.291473 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":131,"thread_start_us":135,"threads_started":1}
I20260806 02:12:54.292083 26391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.284842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.18.70.67:34655)
0806 02:12:54.285160 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.285182 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.285207 (+ 25us) server_negotiation.cc:407] Connection header received
0806 02:12:54.285294 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.285299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.285516 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.285714 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:54.286224 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.288100 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.289305 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.290691 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.290847 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.290989 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.291847 (+ 858us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.291944 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":112,"threads_started":1}
I20260806 02:12:54.292217 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.284380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:41273 (local address 127.0.0.1:37368)
0806 02:12:54.285347 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:54.285361 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:54.285447 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:54.287166 (+ 1719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:54.287170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:54.287189 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:54.287303 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.287309 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.288713 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.288717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.289875 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.289882 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.291010 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.291015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.291201 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.291226 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 02:12:54.291305 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 02:12:54.291473 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":113,"thread_start_us":119,"threads_started":1}
I20260806 02:12:54.293915 26141 tablet_service.cc:1467] Processing CreateTablet for tablet 20d3fd94ed594c21b9a7d83f3512a873 (DEFAULT_TABLE table=test-table [id=6d136ac223d84987a8d23368a37d9d29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:54.294991 26141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d3fd94ed594c21b9a7d83f3512a873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:54.294829 26289 tablet_service.cc:1467] Processing CreateTablet for tablet 20d3fd94ed594c21b9a7d83f3512a873 (DEFAULT_TABLE table=test-table [id=6d136ac223d84987a8d23368a37d9d29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:54.295945 26289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d3fd94ed594c21b9a7d83f3512a873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:54.295670 25999 tablet_service.cc:1467] Processing CreateTablet for tablet 20d3fd94ed594c21b9a7d83f3512a873 (DEFAULT_TABLE table=test-table [id=6d136ac223d84987a8d23368a37d9d29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:54.296684 25999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20d3fd94ed594c21b9a7d83f3512a873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:54.301801 26393 tablet_bootstrap.cc:492] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3: Bootstrap starting.
I20260806 02:12:54.303835 26393 tablet_bootstrap.cc:654] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:54.304353 26394 tablet_bootstrap.cc:492] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c: Bootstrap starting.
I20260806 02:12:54.305145 26393 log.cc:824] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:54.305768 26395 tablet_bootstrap.cc:492] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0: Bootstrap starting.
I20260806 02:12:54.306672 26394 tablet_bootstrap.cc:654] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:54.307274 26393 tablet_bootstrap.cc:492] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3: No bootstrap required, opened a new log
I20260806 02:12:54.307468 26393 ts_tablet_manager.cc:1397] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:12:54.307468 26394 log.cc:824] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:54.308139 26395 tablet_bootstrap.cc:654] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:54.309149 26395 log.cc:824] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:54.309861 26394 tablet_bootstrap.cc:492] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c: No bootstrap required, opened a new log
I20260806 02:12:54.310060 26394 ts_tablet_manager.cc:1397] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260806 02:12:54.311375 26395 tablet_bootstrap.cc:492] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0: No bootstrap required, opened a new log
I20260806 02:12:54.311626 26395 ts_tablet_manager.cc:1397] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:12:54.312114 26393 raft_consensus.cc:348] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } } peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } }
I20260806 02:12:54.312387 26393 raft_consensus.cc:374] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:54.312440 26393 raft_consensus.cc:729] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bad949c005549699519cf75f71049d3, State: Initialized, Role: FOLLOWER
I20260806 02:12:54.312913 26393 consensus_queue.cc:260] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } } peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } }
I20260806 02:12:54.313728 26210 heartbeater.cc:503] Master 127.18.70.126:45685 was elected leader, sending a full tablet report...
I20260806 02:12:54.315433 26393 ts_tablet_manager.cc:1428] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260806 02:12:54.314611 26394 raft_consensus.cc:348] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } } peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } }
I20260806 02:12:54.315912 26394 raft_consensus.cc:374] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:54.315963 26394 raft_consensus.cc:729] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17aeb8b024d0402d8b2282ced5edb90c, State: Initialized, Role: FOLLOWER
I20260806 02:12:54.316409 26394 consensus_queue.cc:260] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } } peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } }
I20260806 02:12:54.316308 26395 raft_consensus.cc:348] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } } peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } }
I20260806 02:12:54.316598 26395 raft_consensus.cc:374] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:54.316674 26395 raft_consensus.cc:729] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15eaf475d67a46b0bc51aa2995c6fca0, State: Initialized, Role: FOLLOWER
I20260806 02:12:54.317138 26395 consensus_queue.cc:260] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } } peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } }
I20260806 02:12:54.317327 26394 ts_tablet_manager.cc:1428] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260806 02:12:54.318089 26395 ts_tablet_manager.cc:1428] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 02:12:54.318573 26358 heartbeater.cc:503] Master 127.18.70.126:45685 was elected leader, sending a full tablet report...
I20260806 02:12:54.320259 26068 heartbeater.cc:503] Master 127.18.70.126:45685 was elected leader, sending a full tablet report...
W20260806 02:12:54.487850 26360 tablet.cc:2369] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 02:12:54.531111 26212 tablet.cc:2369] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:12:54.561219 26400 raft_consensus.cc:482] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:54.561499 26400 raft_consensus.cc:504] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } } peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } }
I20260806 02:12:54.562803 26400 leader_election.cc:288] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bad949c005549699519cf75f71049d3 (127.18.70.66:42507), 15eaf475d67a46b0bc51aa2995c6fca0 (127.18.70.65:41273)
I20260806 02:12:54.567289 26389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.563449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:57049 (local address 127.18.70.66:42507)
0806 02:12:54.563648 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.563655 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.563673 (+ 18us) server_negotiation.cc:407] Connection header received
0806 02:12:54.563745 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.563750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.563841 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.563926 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:54.564371 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.565046 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.566136 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.566555 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.566682 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.566736 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.567058 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.567136 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 02:12:54.567349 26361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.563095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:42507 (local address 127.18.70.67:57049)
0806 02:12:54.563310 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:54.563327 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:54.563423 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:54.564065 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:54.564070 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:54.564092 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:54.564238 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.564247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.565153 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.565158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.566013 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.566022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.566737 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.566742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.566893 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.566932 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 02:12:54.567032 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 02:12:54.567129 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":45}
I20260806 02:12:54.567873 26363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.563058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:41273 (local address 127.18.70.67:42379)
0806 02:12:54.563267 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:54.563283 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:54.563384 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:54.563843 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:54.563846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:54.563862 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:54.563985 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.563990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.564944 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.564949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.566524 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:54.566538 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.567428 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.567431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:54.567586 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.567610 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 02:12:54.567675 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 02:12:54.567748 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":1}
W20260806 02:12:54.567863 26070 tablet.cc:2369] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:12:54.568236 26162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d3fd94ed594c21b9a7d83f3512a873" candidate_uuid: "17aeb8b024d0402d8b2282ced5edb90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bad949c005549699519cf75f71049d3" is_pre_election: true
I20260806 02:12:54.568333 26392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.563253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:42379 (local address 127.18.70.65:41273)
0806 02:12:54.563593 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.563599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.563616 (+ 17us) server_negotiation.cc:407] Connection header received
0806 02:12:54.563673 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.563675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.563713 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.563813 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:54.564095 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.564827 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.566669 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.567169 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.567299 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.567357 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.568127 (+ 770us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.568206 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217}
I20260806 02:12:54.568543 26162 raft_consensus.cc:2454] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17aeb8b024d0402d8b2282ced5edb90c in term 0.
I20260806 02:12:54.569010 26019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d3fd94ed594c21b9a7d83f3512a873" candidate_uuid: "17aeb8b024d0402d8b2282ced5edb90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" is_pre_election: true
I20260806 02:12:54.569057 26236 leader_election.cc:302] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17aeb8b024d0402d8b2282ced5edb90c, 6bad949c005549699519cf75f71049d3; no voters:
I20260806 02:12:54.569314 26019 raft_consensus.cc:2454] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17aeb8b024d0402d8b2282ced5edb90c in term 0.
I20260806 02:12:54.569381 26400 raft_consensus.cc:2781] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:54.569571 26400 raft_consensus.cc:482] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:54.569662 26400 raft_consensus.cc:3037] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:54.570835 26400 raft_consensus.cc:504] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } } peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } }
I20260806 02:12:54.571208 26400 leader_election.cc:288] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [CANDIDATE]: Term 1 election: Requested vote from peers 6bad949c005549699519cf75f71049d3 (127.18.70.66:42507), 15eaf475d67a46b0bc51aa2995c6fca0 (127.18.70.65:41273)
I20260806 02:12:54.571534 26019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d3fd94ed594c21b9a7d83f3512a873" candidate_uuid: "17aeb8b024d0402d8b2282ced5edb90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15eaf475d67a46b0bc51aa2995c6fca0"
I20260806 02:12:54.571575 26162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20d3fd94ed594c21b9a7d83f3512a873" candidate_uuid: "17aeb8b024d0402d8b2282ced5edb90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bad949c005549699519cf75f71049d3"
I20260806 02:12:54.571702 26019 raft_consensus.cc:3037] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:54.571712 26162 raft_consensus.cc:3037] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:54.572822 26019 raft_consensus.cc:2454] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17aeb8b024d0402d8b2282ced5edb90c in term 1.
I20260806 02:12:54.573100 26234 leader_election.cc:302] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15eaf475d67a46b0bc51aa2995c6fca0, 17aeb8b024d0402d8b2282ced5edb90c; no voters:
I20260806 02:12:54.573289 26400 raft_consensus.cc:2781] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:54.573346 26162 raft_consensus.cc:2454] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17aeb8b024d0402d8b2282ced5edb90c in term 1.
I20260806 02:12:54.573551 26400 raft_consensus.cc:686] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [term 1 LEADER]: Becoming Leader. State: Replica: 17aeb8b024d0402d8b2282ced5edb90c, State: Running, Role: LEADER
I20260806 02:12:54.573899 26400 consensus_queue.cc:237] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } } peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } }
I20260806 02:12:54.578749 25046 catalog_manager.cc:5495] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c reported cstate change: term changed from 0 to 1, leader changed from <none> to 17aeb8b024d0402d8b2282ced5edb90c (127.18.70.67). New cstate: current_term: 1 leader_uuid: "17aeb8b024d0402d8b2282ced5edb90c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } health_report { overall_health: UNKNOWN } } }
I20260806 02:12:54.661075 26391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.656365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42982 (local address 127.18.70.67:34655)
0806 02:12:54.656501 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.656505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.656523 (+ 18us) server_negotiation.cc:407] Connection header received
0806 02:12:54.656728 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.656730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.656796 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.656901 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:54.657263 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.658167 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.659219 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.659492 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.659661 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.660038 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:54.660582 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:54.660744 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.660830 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.660897 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 02:12:54.667732 26019 raft_consensus.cc:1261] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 1 FOLLOWER]: Refusing update from remote peer 17aeb8b024d0402d8b2282ced5edb90c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:54.667755 26162 raft_consensus.cc:1261] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3 [term 1 FOLLOWER]: Refusing update from remote peer 17aeb8b024d0402d8b2282ced5edb90c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:54.668514 26402 consensus_queue.cc:1035] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [LEADER]: Connected to new peer: Peer: permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:54.668730 26400 consensus_queue.cc:1035] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:54.675434 26406 mvcc.cc:205] Tried to move back new op lower bound from 7315383806631002112 to 7315383806258683904. Current Snapshot: MvccSnapshot[applied={T|T < 7315383806631002112}]
I20260806 02:12:54.679446 26405 mvcc.cc:205] Tried to move back new op lower bound from 7315383806631002112 to 7315383806258683904. Current Snapshot: MvccSnapshot[applied={T|T < 7315383806631002112}]
I20260806 02:12:54.687819 26407 mvcc.cc:205] Tried to move back new op lower bound from 7315383806631002112 to 7315383806258683904. Current Snapshot: MvccSnapshot[applied={T|T < 7315383806631002112 or (T in {7315383806631002112})}]
I20260806 02:12:54.689162 25046 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:54.689317 25046 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:54.693787 25046 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65c662a093244164b5077e524674f6bd: Sending DeleteTablet for 3 replicas of tablet 20d3fd94ed594c21b9a7d83f3512a873
I20260806 02:12:54.695139 25999 tablet_service.cc:1514] Processing DeleteTablet for tablet 20d3fd94ed594c21b9a7d83f3512a873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:54 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:37368
I20260806 02:12:54.695406 26289 tablet_service.cc:1514] Processing DeleteTablet for tablet 20d3fd94ed594c21b9a7d83f3512a873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:54 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:42976
I20260806 02:12:54.696444 26415 tablet_replica.cc:330] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0: stopping tablet replica
I20260806 02:12:54.696777 26415 raft_consensus.cc:2229] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:54.697196 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 25013
I20260806 02:12:54.698827 26141 tablet_service.cc:1514] Processing DeleteTablet for tablet 20d3fd94ed594c21b9a7d83f3512a873 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:54 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:39374
I20260806 02:12:54.700449 26415 raft_consensus.cc:2258] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:54.702139 26415 ts_tablet_manager.cc:1905] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:54.704936 26415 ts_tablet_manager.cc:1918] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:54.705041 26415 log.cc:1196] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-0/wal/wals/20d3fd94ed594c21b9a7d83f3512a873
I20260806 02:12:54.705384 26415 ts_tablet_manager.cc:1939] T 20d3fd94ed594c21b9a7d83f3512a873 P 15eaf475d67a46b0bc51aa2995c6fca0: Deleting consensus metadata
W20260806 02:12:54.709666 25949 connection.cc:535] server connection from 127.0.0.1:37368 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 02:12:54.710220 26419 tablet_replica.cc:330] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3: stopping tablet replica
I20260806 02:12:54.710623 26419 raft_consensus.cc:2229] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 02:12:54.712386 25670 connection.cc:535] server connection from 127.0.0.1:37636 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 02:12:54.713312 26419 raft_consensus.cc:2258] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:54.717536 26373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.695963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37640 (local address 127.18.70.125:41375)
0806 02:12:54.696084 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.696089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.706831 (+ 10742us) server_negotiation.cc:407] Connection header received
0806 02:12:54.707026 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.707029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.707097 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.707146 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:54.708576 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.709414 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.710651 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.711010 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.711091 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.713027 (+ 1936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:54.713031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:54.713033 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:54.713059 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:54.714710 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.715234 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.715239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.715240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.715323 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.715694 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.715697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.715699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.716022 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:54.716255 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.717264 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.717386 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 02:12:54.717549 26419 ts_tablet_manager.cc:1905] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:54.718466 26418 tablet_replica.cc:330] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c: stopping tablet replica
I20260806 02:12:54.718767 26418 raft_consensus.cc:2229] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:12:54.719136 26418 raft_consensus.cc:2258] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:54.721885 26419 ts_tablet_manager.cc:1918] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:54.722024 26419 log.cc:1196] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/20d3fd94ed594c21b9a7d83f3512a873
I20260806 02:12:54.722430 26419 ts_tablet_manager.cc:1939] T 20d3fd94ed594c21b9a7d83f3512a873 P 6bad949c005549699519cf75f71049d3: Deleting consensus metadata
W20260806 02:12:54.723671 26094 connection.cc:441] server connection from 127.0.0.1:39374 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39374 (request call id 1) could send its response
I20260806 02:12:54.724965 26418 ts_tablet_manager.cc:1905] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:54.727635 26418 ts_tablet_manager.cc:1918] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:54.727736 26418 log.cc:1196] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/20d3fd94ed594c21b9a7d83f3512a873
I20260806 02:12:54.728012 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.716895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37652 (local address 127.18.70.125:41375)
0806 02:12:54.717082 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.717086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.718636 (+ 1550us) server_negotiation.cc:407] Connection header received
0806 02:12:54.718801 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.718804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.718856 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.718904 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:54.720297 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.720984 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.722233 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.722578 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.722657 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.724470 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:54.724473 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:54.724477 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:54.724504 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:54.726197 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.726668 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.726671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.726673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.726755 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.727054 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.727057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.727059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.727245 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:54.727353 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.727707 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.727818 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 02:12:54.728048 26418 ts_tablet_manager.cc:1939] T 20d3fd94ed594c21b9a7d83f3512a873 P 17aeb8b024d0402d8b2282ced5edb90c: Deleting consensus metadata
W20260806 02:12:54.729007 26235 connection.cc:441] server connection from 127.0.0.1:42976 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42976 (request call id 1) could send its response
I20260806 02:12:54.733726 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.717384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58016 (local address 127.18.70.124:45749)
0806 02:12:54.717717 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.717722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.724099 (+ 6377us) server_negotiation.cc:407] Connection header received
0806 02:12:54.724279 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.724283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.724356 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.724412 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:54.725476 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.726273 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.727394 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.727737 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.727811 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.730131 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:54.730134 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:54.730137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:54.730161 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:54.731941 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.732528 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.732531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.732532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.732610 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.732902 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.732907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.732908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.733084 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:54.733221 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.733440 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.733525 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":259,"spinlock_wait_cycles":768}
I20260806 02:12:54.734115 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:54.696057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.18.70.124:45749)
0806 02:12:54.696190 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:54.696196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:54.718306 (+ 22110us) server_negotiation.cc:407] Connection header received
0806 02:12:54.718379 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:54.718384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:54.718452 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:54.718539 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:54.720110 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.720889 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.723705 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:54.724063 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:54.724152 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:54.730293 (+ 6141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:54.730296 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:54.730299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:54.730323 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:54.732043 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.732464 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.732467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.732469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.732551 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:54.732854 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:54.732857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:54.732859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:54.733087 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:54.733221 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:54.733866 (+ 645us) server_negotiation.cc:299] Negotiation successful
0806 02:12:54.733981 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":45}
I20260806 02:12:55.680462 26427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:55.679980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.65:56301)
0806 02:12:55.680228 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:55.680339 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":95,"threads_started":1}
W20260806 02:12:55.680792 26068 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:45685 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
I20260806 02:12:55.685355 26428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:55.684873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.67:38429)
0806 02:12:55.685132 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:55.685233 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":90,"threads_started":1}
W20260806 02:12:55.686038 26358 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:45685 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
I20260806 02:12:55.686120 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:55.685637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.66:47673)
0806 02:12:55.685910 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:55.686003 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":73,"threads_started":1}
W20260806 02:12:55.686407 26210 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:45685 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
I20260806 02:12:56.192970 26430 raft_consensus.cc:482] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 65c662a093244164b5077e524674f6bd)
I20260806 02:12:56.193173 26430 raft_consensus.cc:504] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } }
I20260806 02:12:56.194165 26430 leader_election.cc:288] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 65c662a093244164b5077e524674f6bd (127.18.70.126:45685), a71d40aa02ae4b23980e6b02c76ab24a (127.18.70.124:45749)
I20260806 02:12:56.194574 26432 raft_consensus.cc:482] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 65c662a093244164b5077e524674f6bd)
I20260806 02:12:56.194795 26432 raft_consensus.cc:504] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } }
I20260806 02:12:56.194913 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.194338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46606)
0806 02:12:56.194581 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.194787 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":82,"threads_started":1}
W20260806 02:12:56.195320 25668 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
I20260806 02:12:56.196342 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.196107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46614)
0806 02:12:56.196228 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.196277 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 02:12:56.196393 26437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.195936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46612)
0806 02:12:56.196228 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.196306 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":76,"threads_started":1}
I20260806 02:12:56.196460 26432 leader_election.cc:288] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 65c662a093244164b5077e524674f6bd (127.18.70.126:45685), a686923b459746f0acebe48c9100cc25 (127.18.70.125:41375)
W20260806 02:12:56.196544 25865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
W20260806 02:12:56.196623 25668 leader_election.cc:334] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 65c662a093244164b5077e524674f6bd (127.18.70.126:45685): Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
I20260806 02:12:56.197737 26437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.197512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46622)
0806 02:12:56.197630 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.197679 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 02:12:56.197932 25865 leader_election.cc:334] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 65c662a093244164b5077e524674f6bd (127.18.70.126:45685): Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
I20260806 02:12:56.199028 26434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.194876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58020 (local address 127.18.70.124:45749)
0806 02:12:56.195099 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:56.195103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:56.195334 (+ 231us) server_negotiation.cc:407] Connection header received
0806 02:12:56.195392 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:56.195395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:56.195444 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:56.195529 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:56.195861 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.196645 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.197985 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.198509 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.198628 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.198689 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:56.198848 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 02:12:56.198923 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":157,"thread_start_us":76,"threads_started":1}
I20260806 02:12:56.199038 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.194763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:45749 (local address 127.0.0.1:58020)
0806 02:12:56.195094 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.195110 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:56.195221 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:56.195582 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:56.195585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:56.195609 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:56.195719 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.195726 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.196775 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.196778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.197876 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.197886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.198629 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.198633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.198722 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.198750 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 02:12:56.198832 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 02:12:56.198905 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":82,"threads_started":1}
I20260806 02:12:56.199513 25885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a686923b459746f0acebe48c9100cc25" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" is_pre_election: true
I20260806 02:12:56.199759 25885 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a686923b459746f0acebe48c9100cc25 in term 1.
I20260806 02:12:56.200140 25668 leader_election.cc:302] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a686923b459746f0acebe48c9100cc25, a71d40aa02ae4b23980e6b02c76ab24a; no voters: 65c662a093244164b5077e524674f6bd
I20260806 02:12:56.200278 26436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.196031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:41375 (local address 127.0.0.1:37662)
0806 02:12:56.196652 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.196665 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:56.196744 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:56.197187 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:56.197191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:56.197210 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:56.197292 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.197298 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.198216 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.198221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.199068 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.199099 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.199864 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.199868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.199973 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.199997 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 02:12:56.200083 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 02:12:56.200151 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":68,"threads_started":1}
I20260806 02:12:56.200366 26430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 02:12:56.200428 26430 raft_consensus.cc:482] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 65c662a093244164b5077e524674f6bd)
I20260806 02:12:56.200466 26430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 1 FOLLOWER]: Advancing to term 2
I20260806 02:12:56.200621 26439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.196294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.18.70.125:41375)
0806 02:12:56.196995 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:56.197002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:56.197012 (+ 10us) server_negotiation.cc:407] Connection header received
0806 02:12:56.197045 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:56.197046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:56.197076 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:56.197145 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:56.197417 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.198106 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.199208 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.199734 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.199862 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.199914 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:56.200472 (+ 558us) server_negotiation.cc:299] Negotiation successful
0806 02:12:56.200544 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":92,"threads_started":1}
I20260806 02:12:56.200986 25693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a686923b459746f0acebe48c9100cc25" is_pre_election: true
I20260806 02:12:56.201637 26430 raft_consensus.cc:504] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } }
I20260806 02:12:56.201859 25693 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a71d40aa02ae4b23980e6b02c76ab24a in current term 2: Already voted for candidate a686923b459746f0acebe48c9100cc25 in this term.
I20260806 02:12:56.201982 26430 leader_election.cc:288] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [CANDIDATE]: Term 2 election: Requested vote from peers 65c662a093244164b5077e524674f6bd (127.18.70.126:45685), a71d40aa02ae4b23980e6b02c76ab24a (127.18.70.124:45749)
I20260806 02:12:56.202284 25864 leader_election.cc:302] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a71d40aa02ae4b23980e6b02c76ab24a; no voters: 65c662a093244164b5077e524674f6bd, a686923b459746f0acebe48c9100cc25
I20260806 02:12:56.202337 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.202110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46632)
0806 02:12:56.202216 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.202273 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 02:12:56.202513 26432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 1 FOLLOWER]: Advancing to term 2
I20260806 02:12:56.202453 25885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a686923b459746f0acebe48c9100cc25" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a71d40aa02ae4b23980e6b02c76ab24a"
I20260806 02:12:56.202929 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.202732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46636)
0806 02:12:56.202811 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.202857 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 02:12:56.203023 25668 leader_election.cc:334] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 65c662a093244164b5077e524674f6bd (127.18.70.126:45685): Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
I20260806 02:12:56.203534 26432 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:12:56.204543 25885 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a686923b459746f0acebe48c9100cc25 in term 2.
I20260806 02:12:56.205148 25668 leader_election.cc:302] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a686923b459746f0acebe48c9100cc25, a71d40aa02ae4b23980e6b02c76ab24a; no voters: 65c662a093244164b5077e524674f6bd
I20260806 02:12:56.205302 26430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 02:12:56.205458 26430 raft_consensus.cc:686] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [term 2 LEADER]: Becoming Leader. State: Replica: a686923b459746f0acebe48c9100cc25, State: Running, Role: LEADER
I20260806 02:12:56.205688 26430 consensus_queue.cc:237] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } }
I20260806 02:12:56.206667 26441 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a686923b459746f0acebe48c9100cc25. Latest consensus state: current_term: 2 leader_uuid: "a686923b459746f0acebe48c9100cc25" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } } }
I20260806 02:12:56.206796 26441 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:56.671108 25885 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [term 2 FOLLOWER]: Refusing update from remote peer a686923b459746f0acebe48c9100cc25: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20260806 02:12:56.671682 26441 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 02:12:56.672945 26432 sys_catalog.cc:455] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a686923b459746f0acebe48c9100cc25. Latest consensus state: current_term: 2 leader_uuid: "a686923b459746f0acebe48c9100cc25" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } } }
I20260806 02:12:56.673061 26432 sys_catalog.cc:458] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:12:56.674376 26441 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a686923b459746f0acebe48c9100cc25" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } } }
I20260806 02:12:56.674521 26441 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:56.674594 26432 sys_catalog.cc:455] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a686923b459746f0acebe48c9100cc25" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "65c662a093244164b5077e524674f6bd" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45685 } } peers { permanent_uuid: "a686923b459746f0acebe48c9100cc25" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 41375 } attrs { promote: false } } peers { permanent_uuid: "a71d40aa02ae4b23980e6b02c76ab24a" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 45749 } attrs { promote: false } } }
I20260806 02:12:56.674712 26432 sys_catalog.cc:458] T 00000000000000000000000000000000 P a71d40aa02ae4b23980e6b02c76ab24a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:12:56.674901 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.674617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46716)
0806 02:12:56.674736 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.674817 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 02:12:56.675619 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.675369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.0.0.1:46726)
0806 02:12:56.675473 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.675528 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 02:12:56.675733 25668 consensus_peers.cc:487] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25 -> Peer 65c662a093244164b5077e524674f6bd (127.18.70.126:45685): Couldn't send request to peer 65c662a093244164b5077e524674f6bd. Status: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 02:12:56.681612 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.681231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.65:36379)
0806 02:12:56.681474 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.681531 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":99,"threads_started":1}
I20260806 02:12:56.687121 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.686648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.67:51993)
0806 02:12:56.686946 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.686997 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":77,"threads_started":1}
I20260806 02:12:56.687460 26451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.686933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.66:41483)
0806 02:12:56.687298 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.687364 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":76,"threads_started":1}
I20260806 02:12:56.722819 26443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:56.725385 26443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:56.725746 26443 catalog_manager.cc:1259] Loaded cluster ID: 93d9857deab442bda1973b5747078e73
I20260806 02:12:56.725800 26443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:56.726306 26443 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:56.726644 26443 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: Loaded TSK: 0
I20260806 02:12:56.727299 26443 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:12:56.787212 25682 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:12:56.790561 25682 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:56.804541 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.800233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:34655 (local address 127.0.0.1:42996)
0806 02:12:56.800663 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.800681 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:56.800759 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:56.801040 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:56.801043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:56.801063 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:56.801168 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.801174 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.802168 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.802171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.803142 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.803151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.804093 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.804098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.804237 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.804270 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 02:12:56.804333 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 02:12:56.804405 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":28}
I20260806 02:12:56.805169 26453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.800359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.18.70.67:34655)
0806 02:12:56.800767 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:56.800772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:56.800786 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:12:56.800866 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:56.800868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:56.800926 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:56.801017 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:56.801304 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.802068 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.803289 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.803959 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.804812 (+ 853us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.804885 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:56.804934 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 02:12:56.805008 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":106,"threads_started":1}
I20260806 02:12:56.805529 26454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.800693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:42507 (local address 127.0.0.1:39380)
0806 02:12:56.801188 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.801202 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:56.801288 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:56.801828 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:56.801831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:56.801849 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:56.801938 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.801945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.802872 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.802880 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.804083 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.804091 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.804997 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.805000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.805103 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.805128 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 02:12:56.805193 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 02:12:56.805374 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":1,"thread_start_us":76,"threads_started":1}
I20260806 02:12:56.805624 26457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.801351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37380 (local address 127.18.70.65:41273)
0806 02:12:56.801838 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:56.801843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:56.801857 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:12:56.801908 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:56.801910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:56.801956 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:56.802041 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:56.802427 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.803088 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.804414 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.805026 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.805130 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.805212 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:56.805394 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 02:12:56.805484 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":69,"threads_started":1}
I20260806 02:12:56.805682 26289 tablet_service.cc:1467] Processing CreateTablet for tablet ad4b9f6cded44813a8814df99ac8a542 (DEFAULT_TABLE table=test-table [id=9ef03e34611a4b42929aeefd3afa4323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:56.805841 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.800819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39380 (local address 127.18.70.66:42507)
0806 02:12:56.801610 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:56.801615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:56.801629 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:12:56.801680 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:56.801683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:56.801728 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:56.801822 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:56.802056 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.802746 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.804211 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.804857 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.804959 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.805015 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:56.805494 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 02:12:56.805569 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":87,"threads_started":1}
I20260806 02:12:56.805850 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.801017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:41273 (local address 127.0.0.1:37380)
0806 02:12:56.801587 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.801600 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:56.801676 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:56.802165 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:56.802168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:56.802184 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:56.802303 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.802309 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.803193 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.803198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.804272 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.804283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.805146 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.805150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.805259 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.805282 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 02:12:56.805356 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 02:12:56.805430 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":68,"thread_start_us":65,"threads_started":1}
I20260806 02:12:56.806023 26289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad4b9f6cded44813a8814df99ac8a542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:56.806340 26141 tablet_service.cc:1467] Processing CreateTablet for tablet ad4b9f6cded44813a8814df99ac8a542 (DEFAULT_TABLE table=test-table [id=9ef03e34611a4b42929aeefd3afa4323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:56.806581 26141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad4b9f6cded44813a8814df99ac8a542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:56.808660 26458 tablet_bootstrap.cc:492] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c: Bootstrap starting.
I20260806 02:12:56.809196 25999 tablet_service.cc:1467] Processing CreateTablet for tablet ad4b9f6cded44813a8814df99ac8a542 (DEFAULT_TABLE table=test-table [id=9ef03e34611a4b42929aeefd3afa4323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:56.809419 25999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad4b9f6cded44813a8814df99ac8a542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:56.809756 26459 tablet_bootstrap.cc:492] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3: Bootstrap starting.
I20260806 02:12:56.809787 26458 tablet_bootstrap.cc:654] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:56.810555 26459 tablet_bootstrap.cc:654] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:56.811741 26458 tablet_bootstrap.cc:492] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c: No bootstrap required, opened a new log
I20260806 02:12:56.811916 26458 ts_tablet_manager.cc:1397] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:12:56.811959 26459 tablet_bootstrap.cc:492] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3: No bootstrap required, opened a new log
I20260806 02:12:56.812047 26459 ts_tablet_manager.cc:1397] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 02:12:56.812441 26458 raft_consensus.cc:348] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } } peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } }
I20260806 02:12:56.812461 26459 raft_consensus.cc:348] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } } peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } }
I20260806 02:12:56.812579 26459 raft_consensus.cc:374] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:56.812579 26458 raft_consensus.cc:374] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:56.812613 26458 raft_consensus.cc:729] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17aeb8b024d0402d8b2282ced5edb90c, State: Initialized, Role: FOLLOWER
I20260806 02:12:56.812613 26459 raft_consensus.cc:729] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bad949c005549699519cf75f71049d3, State: Initialized, Role: FOLLOWER
I20260806 02:12:56.812740 26458 consensus_queue.cc:260] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } } peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } }
I20260806 02:12:56.812740 26459 consensus_queue.cc:260] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } } peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } }
I20260806 02:12:56.813105 26459 ts_tablet_manager.cc:1428] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 02:12:56.813160 26458 ts_tablet_manager.cc:1428] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:56.813679 26451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.813427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.66:35409)
0806 02:12:56.813552 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.813608 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 02:12:56.813809 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.813564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.67:41995)
0806 02:12:56.813672 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.813727 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 02:12:56.814509 26451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.814185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.66:45463)
0806 02:12:56.814394 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.814442 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260806 02:12:56.817157 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.816896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.67:37945)
0806 02:12:56.817018 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.817079 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 02:12:56.818398 26359 heartbeater.cc:503] Master 127.18.70.125:41375 was elected leader, sending a full tablet report...
I20260806 02:12:56.819849 26211 heartbeater.cc:503] Master 127.18.70.125:41375 was elected leader, sending a full tablet report...
I20260806 02:12:56.819993 26462 tablet_bootstrap.cc:492] T ad4b9f6cded44813a8814df99ac8a542 P 15eaf475d67a46b0bc51aa2995c6fca0: Bootstrap starting.
I20260806 02:12:56.820839 26462 tablet_bootstrap.cc:654] T ad4b9f6cded44813a8814df99ac8a542 P 15eaf475d67a46b0bc51aa2995c6fca0: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:56.822283 26462 tablet_bootstrap.cc:492] T ad4b9f6cded44813a8814df99ac8a542 P 15eaf475d67a46b0bc51aa2995c6fca0: No bootstrap required, opened a new log
I20260806 02:12:56.822391 26462 ts_tablet_manager.cc:1397] T ad4b9f6cded44813a8814df99ac8a542 P 15eaf475d67a46b0bc51aa2995c6fca0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 02:12:56.822811 26462 raft_consensus.cc:348] T ad4b9f6cded44813a8814df99ac8a542 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } } peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } }
I20260806 02:12:56.822925 26462 raft_consensus.cc:374] T ad4b9f6cded44813a8814df99ac8a542 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:56.822970 26462 raft_consensus.cc:729] T ad4b9f6cded44813a8814df99ac8a542 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15eaf475d67a46b0bc51aa2995c6fca0, State: Initialized, Role: FOLLOWER
I20260806 02:12:56.823110 26462 consensus_queue.cc:260] T ad4b9f6cded44813a8814df99ac8a542 P 15eaf475d67a46b0bc51aa2995c6fca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } } peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } }
I20260806 02:12:56.823443 26462 ts_tablet_manager.cc:1428] T ad4b9f6cded44813a8814df99ac8a542 P 15eaf475d67a46b0bc51aa2995c6fca0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:56.824175 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.823953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.65:55717)
0806 02:12:56.824053 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.824111 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 02:12:56.824877 26069 heartbeater.cc:503] Master 127.18.70.125:41375 was elected leader, sending a full tablet report...
I20260806 02:12:56.825117 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.824895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.65:50457)
0806 02:12:56.825009 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.825060 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 02:12:56.960295 26463 raft_consensus.cc:482] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:56.960438 26463 raft_consensus.cc:504] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } } peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } }
I20260806 02:12:56.961674 26463 leader_election.cc:288] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17aeb8b024d0402d8b2282ced5edb90c (127.18.70.67:34655), 15eaf475d67a46b0bc51aa2995c6fca0 (127.18.70.65:41273)
I20260806 02:12:56.965889 26451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.961888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:34655 (local address 127.18.70.66:44499)
0806 02:12:56.962079 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.962097 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:56.962219 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:56.962517 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:56.962522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:56.962545 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:56.962705 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.962710 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.963606 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.963610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.964477 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.964486 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.965361 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.965368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.965540 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.965581 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 02:12:56.965657 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 02:12:56.965755 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 02:12:56.965984 26457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.962208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:36919 (local address 127.18.70.65:41273)
0806 02:12:56.962336 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:56.962340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:56.962582 (+ 242us) server_negotiation.cc:407] Connection header received
0806 02:12:56.962645 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:56.962647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:56.962685 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:56.962767 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:56.963061 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.963596 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.964651 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.965315 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.965440 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.965492 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:56.965782 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 02:12:56.965860 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 02:12:56.966362 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.961921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:41273 (local address 127.18.70.66:36919)
0806 02:12:56.962446 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.962462 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:56.962578 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:56.962806 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:56.962811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:56.962832 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:56.962929 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.962934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.963722 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.963727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.964513 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:56.964523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.965548 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.965552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:56.965652 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.965671 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 02:12:56.966114 (+ 443us) client_negotiation.cc:240] Negotiation successful
0806 02:12:56.966201 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":190,"threads_started":1}
I20260806 02:12:56.966501 26453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.962013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:44499 (local address 127.18.70.67:34655)
0806 02:12:56.962178 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:56.962183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:56.962199 (+ 16us) server_negotiation.cc:407] Connection header received
0806 02:12:56.962331 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:56.962335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:56.962395 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:56.962459 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:56.962855 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.963466 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.964620 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.965207 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.965354 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.965423 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:56.965716 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 02:12:56.965878 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 02:12:56.966832 26019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad4b9f6cded44813a8814df99ac8a542" candidate_uuid: "6bad949c005549699519cf75f71049d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" is_pre_election: true
I20260806 02:12:56.967011 26019 raft_consensus.cc:2454] T ad4b9f6cded44813a8814df99ac8a542 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bad949c005549699519cf75f71049d3 in term 0.
I20260806 02:12:56.966962 26309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad4b9f6cded44813a8814df99ac8a542" candidate_uuid: "6bad949c005549699519cf75f71049d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17aeb8b024d0402d8b2282ced5edb90c" is_pre_election: true
I20260806 02:12:56.967213 26309 raft_consensus.cc:2454] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bad949c005549699519cf75f71049d3 in term 0.
I20260806 02:12:56.967483 26093 leader_election.cc:302] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15eaf475d67a46b0bc51aa2995c6fca0, 6bad949c005549699519cf75f71049d3; no voters:
I20260806 02:12:56.967868 26463 raft_consensus.cc:2781] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:56.967929 26463 raft_consensus.cc:482] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:56.967973 26463 raft_consensus.cc:3037] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:56.969036 26463 raft_consensus.cc:504] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } } peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } }
I20260806 02:12:56.969429 26463 leader_election.cc:288] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [CANDIDATE]: Term 1 election: Requested vote from peers 17aeb8b024d0402d8b2282ced5edb90c (127.18.70.67:34655), 15eaf475d67a46b0bc51aa2995c6fca0 (127.18.70.65:41273)
I20260806 02:12:56.969729 26309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad4b9f6cded44813a8814df99ac8a542" candidate_uuid: "6bad949c005549699519cf75f71049d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17aeb8b024d0402d8b2282ced5edb90c"
I20260806 02:12:56.969794 26019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad4b9f6cded44813a8814df99ac8a542" candidate_uuid: "6bad949c005549699519cf75f71049d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15eaf475d67a46b0bc51aa2995c6fca0"
I20260806 02:12:56.969861 26309 raft_consensus.cc:3037] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:56.969924 26019 raft_consensus.cc:3037] T ad4b9f6cded44813a8814df99ac8a542 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:56.970835 26019 raft_consensus.cc:2454] T ad4b9f6cded44813a8814df99ac8a542 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bad949c005549699519cf75f71049d3 in term 1.
I20260806 02:12:56.970849 26309 raft_consensus.cc:2454] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bad949c005549699519cf75f71049d3 in term 1.
I20260806 02:12:56.971135 26095 leader_election.cc:302] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17aeb8b024d0402d8b2282ced5edb90c, 6bad949c005549699519cf75f71049d3; no voters:
I20260806 02:12:56.971338 26463 raft_consensus.cc:2781] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:56.971617 26463 raft_consensus.cc:686] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [term 1 LEADER]: Becoming Leader. State: Replica: 6bad949c005549699519cf75f71049d3, State: Running, Role: LEADER
I20260806 02:12:56.971868 26463 consensus_queue.cc:237] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } } peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } }
I20260806 02:12:56.972281 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.972028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.66:55283)
0806 02:12:56.972152 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:56.972207 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 02:12:56.974196 25682 catalog_manager.cc:5495] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bad949c005549699519cf75f71049d3 (127.18.70.66). New cstate: current_term: 1 leader_uuid: "6bad949c005549699519cf75f71049d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bad949c005549699519cf75f71049d3" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42507 } health_report { overall_health: HEALTHY } } }
I20260806 02:12:56.997788 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:56.994209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39384 (local address 127.18.70.66:42507)
0806 02:12:56.994354 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:56.994358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:56.994484 (+ 126us) server_negotiation.cc:407] Connection header received
0806 02:12:56.994678 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:56.994681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:56.994732 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:56.994825 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:56.995071 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.995655 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.996543 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:56.996743 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:56.996819 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:56.997076 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:56.997505 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:56.997588 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:56.997647 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 02:12:56.997691 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 02:12:57.000332 26019 raft_consensus.cc:1261] T ad4b9f6cded44813a8814df99ac8a542 P 15eaf475d67a46b0bc51aa2995c6fca0 [term 1 FOLLOWER]: Refusing update from remote peer 6bad949c005549699519cf75f71049d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:57.000605 26309 raft_consensus.cc:1261] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c [term 1 FOLLOWER]: Refusing update from remote peer 6bad949c005549699519cf75f71049d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:57.000905 26469 consensus_queue.cc:1035] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15eaf475d67a46b0bc51aa2995c6fca0" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:57.001168 26463 consensus_queue.cc:1035] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17aeb8b024d0402d8b2282ced5edb90c" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:57.002780 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:57.002493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.65:42111)
0806 02:12:57.002629 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:57.002696 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 02:12:57.004086 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:57.003839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.67:60469)
0806 02:12:57.003962 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:57.004018 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 02:12:57.007068 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:57.006831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.65:44927)
0806 02:12:57.006943 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:57.007001 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 02:12:57.007727 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:57.007243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.67:50257)
0806 02:12:57.007535 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:57.007645 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225}
I20260806 02:12:57.008876 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:57.008225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.66:53555)
0806 02:12:57.008738 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:57.008806 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450}
I20260806 02:12:57.009716 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:57.009498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45685 (local address 127.18.70.66:57533)
0806 02:12:57.009607 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:57.009657 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 02:12:57.018064 25682 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:57.018260 25682 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:57.022571 25682 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a686923b459746f0acebe48c9100cc25: Sending DeleteTablet for 3 replicas of tablet ad4b9f6cded44813a8814df99ac8a542
I20260806 02:12:57.023432 26289 tablet_service.cc:1514] Processing DeleteTablet for tablet ad4b9f6cded44813a8814df99ac8a542 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:57 UTC) from {username='oryx', principal='oryx/127.18.70.125@KRBTEST.COM'} at 127.0.0.1:42996
I20260806 02:12:57.023676 26141 tablet_service.cc:1514] Processing DeleteTablet for tablet ad4b9f6cded44813a8814df99ac8a542 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:57 UTC) from {username='oryx', principal='oryx/127.18.70.125@KRBTEST.COM'} at 127.0.0.1:39380
I20260806 02:12:57.024122 26483 tablet_replica.cc:330] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3: stopping tablet replica
I20260806 02:12:57.024291 26483 raft_consensus.cc:2229] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:12:57.024405 25999 tablet_service.cc:1514] Processing DeleteTablet for tablet ad4b9f6cded44813a8814df99ac8a542 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:57 UTC) from {username='oryx', principal='oryx/127.18.70.125@KRBTEST.COM'} at 127.0.0.1:37380
I20260806 02:12:57.024664 26482 tablet_replica.cc:330] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c: stopping tablet replica
I20260806 02:12:57.024845 26482 raft_consensus.cc:2229] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:57.024878 26483 raft_consensus.cc:2258] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:57.025009 26482 raft_consensus.cc:2258] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:57.025647 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 25838
I20260806 02:12:57.026655 26483 ts_tablet_manager.cc:1905] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:57.029150 26482 ts_tablet_manager.cc:1905] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:57.032188 26482 ts_tablet_manager.cc:1918] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:57.032271 26482 log.cc:1196] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/ad4b9f6cded44813a8814df99ac8a542
I20260806 02:12:57.032639 26482 ts_tablet_manager.cc:1939] T ad4b9f6cded44813a8814df99ac8a542 P 17aeb8b024d0402d8b2282ced5edb90c: Deleting consensus metadata
I20260806 02:12:57.033432 25670 catalog_manager.cc:4841] TS 17aeb8b024d0402d8b2282ced5edb90c (127.18.70.67:34655): tablet ad4b9f6cded44813a8814df99ac8a542 (table test-table [id=9ef03e34611a4b42929aeefd3afa4323]) successfully deleted
W20260806 02:12:57.037250 26456 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:12:57.036882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:41273 (local address 127.0.0.1:37392)
0806 02:12:57.037082 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:57.037163 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:41273: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 02:12:57.037262 26483 ts_tablet_manager.cc:1918] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:57.037344 26483 log.cc:1196] T ad4b9f6cded44813a8814df99ac8a542 P 6bad949c005549699519cf75f71049d3: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/ad4b9f6cded44813a8814df99ac8a542
I20260806 02:12:57.037391 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 26075
W20260806 02:12:57.037478 25667 catalog_manager.cc:4584] TS 15eaf475d67a46b0bc51aa2995c6fca0 (127.18.70.65:41273): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad4b9f6cded44813a8814df99ac8a542: Network error: Client connection negotiation failed: client connection to 127.18.70.65:41273: connect: Connection reset by peer (error 104)
I20260806 02:12:57.048077 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 26216
I20260806 02:12:57.048362 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:57.048031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:42507 (local address 127.0.0.1:39398)
0806 02:12:57.048192 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:57.048262 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.66:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 02:12:57.056636 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 25535
I20260806 02:12:57.063791 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 25652
2026-08-06T02:12:57Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6154 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:12:57 dist-test-slave-6k22 krb5kdc[26487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:12:57 dist-test-slave-6k22 krb5kdc[26487](info): set up 2 sockets
Aug 06 02:12:57 dist-test-slave-6k22 krb5kdc[26487](info): commencing operation
krb5kdc: starting...
W20260806 02:12:58.179625 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:12:58 dist-test-slave-6k22 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:12:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.126.keytab.
Entry for principal oryx/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.126.keytab.
I20260806 02:12:58.306344 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:44033
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.126.keytab
--principal=oryx/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:44033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:58.422168 26500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:58.422472 26500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:58.422514 26500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:58.426448 26500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:12:58.426494 26500 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 02:12:58.426522 26500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:58.426538 26500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:58.426571 26500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:12:58.426586 26500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:12:58.431497 26500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:44033
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.126.keytab
--principal=oryx/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:44033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:58.432590 26500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:58.433560 26500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:58.442243 26505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:58.442261 26508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:58.442268 26506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:58.442719 26500 server_base.cc:1034] running on GCE node
I20260806 02:12:58.443305 26500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:58.444571 26500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:58.445780 26500 hybrid_clock.cc:648] HybridClock initialized: now 1785982378445763 us; error 36 us; skew 500 ppm
Aug 06 02:12:58 dist-test-slave-6k22 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982378, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:58.448807 26500 init.cc:377] Logged in from keytab as oryx/127.18.70.126@KRBTEST.COM (short username oryx)
I20260806 02:12:58.449604 26500 webserver.cc:460] Webserver started at http://127.18.70.126:43663/ using document root <none> and password file <none>
I20260806 02:12:58.450142 26500 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:58.450230 26500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:58.450467 26500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:58.452473 26500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "ce1b120a62474933b5ec52ffb8b7e040"
format_stamp: "Formatted at 2026-08-06 02:12:58 on dist-test-slave-6k22"
server_key: "50cf04773bcdbb4bc232f5baee372ca8"
server_key_iv: "a8a80d92eec03324f356d56bd63fc276"
server_key_version: "encryptionkey@0"
I20260806 02:12:58.452997 26500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "ce1b120a62474933b5ec52ffb8b7e040"
format_stamp: "Formatted at 2026-08-06 02:12:58 on dist-test-slave-6k22"
server_key: "50cf04773bcdbb4bc232f5baee372ca8"
server_key_iv: "a8a80d92eec03324f356d56bd63fc276"
server_key_version: "encryptionkey@0"
I20260806 02:12:58.456625 26500 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 02:12:58.458943 26517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:58.460196 26500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:12:58.460333 26500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "ce1b120a62474933b5ec52ffb8b7e040"
format_stamp: "Formatted at 2026-08-06 02:12:58 on dist-test-slave-6k22"
server_key: "50cf04773bcdbb4bc232f5baee372ca8"
server_key_iv: "a8a80d92eec03324f356d56bd63fc276"
server_key_version: "encryptionkey@0"
I20260806 02:12:58.460439 26500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:58.478259 26500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:58.480999 26500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:58.481204 26500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:58.488994 26500 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:44033
I20260806 02:12:58.489060 26586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:44033 every 8 connection(s)
I20260806 02:12:58.489934 26500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:12:58.492635 26587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:58.493604 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 26500
I20260806 02:12:58.493738 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:12:58.494074 18713 external_mini_cluster.cc:1428] Setting key 7ae52e5d11e79161e818df90c41d0682
I20260806 02:12:58.500020 26587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040: Bootstrap starting.
I20260806 02:12:58.502496 26587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:58.503216 26587 log.cc:824] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040: Log is configured to *not* fsync() on all Append() calls
Aug 06 02:12:58 dist-test-slave-6k22 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
I20260806 02:12:58.505309 26587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040: No bootstrap required, opened a new log
I20260806 02:12:58.509068 26590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:58.495852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53392 (local address 127.18.70.126:44033)
0806 02:12:58.496340 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:58.496359 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:58.496403 (+ 44us) server_negotiation.cc:407] Connection header received
0806 02:12:58.497244 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:58.497269 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:58.497776 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:58.498150 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:58.498829 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:58.500292 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:58.501107 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:58.501758 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:58.501908 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:58.504515 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:58.504520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:58.504533 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:58.504557 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:58.506331 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:58.506805 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:58.506810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:58.506814 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:58.506920 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:58.507194 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:58.507196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:58.507197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:58.507596 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:58.507769 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:58.508034 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 02:12:58.508343 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":133,"threads_started":1}
I20260806 02:12:58.509192 26587 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1b120a62474933b5ec52ffb8b7e040" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44033 } }
I20260806 02:12:58.509457 26587 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:58.509533 26587 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1b120a62474933b5ec52ffb8b7e040, State: Initialized, Role: FOLLOWER
I20260806 02:12:58.509963 26587 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1b120a62474933b5ec52ffb8b7e040" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44033 } }
I20260806 02:12:58.510114 26587 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:12:58.510177 26587 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:12:58.510285 26587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:58.511127 26587 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1b120a62474933b5ec52ffb8b7e040" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44033 } }
I20260806 02:12:58.511492 26587 leader_election.cc:302] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce1b120a62474933b5ec52ffb8b7e040; no voters:
I20260806 02:12:58.511845 26587 leader_election.cc:288] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:12:58.511950 26592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:58.512192 26592 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [term 1 LEADER]: Becoming Leader. State: Replica: ce1b120a62474933b5ec52ffb8b7e040, State: Running, Role: LEADER
I20260806 02:12:58.512799 26592 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1b120a62474933b5ec52ffb8b7e040" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44033 } }
I20260806 02:12:58.513135 26587 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:12:58.514370 26593 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce1b120a62474933b5ec52ffb8b7e040" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1b120a62474933b5ec52ffb8b7e040" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44033 } } }
I20260806 02:12:58.514493 26593 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [sys.catalog]: This master's current role is: LEADER
I20260806 02:12:58.514855 26594 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce1b120a62474933b5ec52ffb8b7e040. Latest consensus state: current_term: 1 leader_uuid: "ce1b120a62474933b5ec52ffb8b7e040" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1b120a62474933b5ec52ffb8b7e040" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44033 } } }
I20260806 02:12:58.515002 26594 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040 [sys.catalog]: This master's current role is: LEADER
W20260806 02:12:58.517465 26608 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:12:58.517551 26608 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:12:58.517620 26603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:12:58.518306 26603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:12:58.523650 26603 catalog_manager.cc:1347] Generated new cluster ID: 0ba68d56e53549679369683b80818a9b
I20260806 02:12:58.523711 26603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:12:58.546103 26603 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:12:58.547430 26603 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:12:58.558791 26603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040: Generated new TSK 0
I20260806 02:12:58.559715 26603 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab.
Entry for principal oryx/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab.
I20260806 02:12:58.594003 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:44033
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab
--principal=oryx/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:58.705317 26615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:58.705601 26615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:58.705680 26615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:58.709729 26615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:58.709807 26615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:58.709877 26615 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 02:12:58.709950 26615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:12:58.715286 26615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab
--principal=oryx/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.70.126:44033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:58.716317 26615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:58.717301 26615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:58.726828 26622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:58.726866 26624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:58.727072 26615 server_base.cc:1034] running on GCE node
W20260806 02:12:58.726840 26621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:58.727546 26615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:58.728231 26615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:58.729414 26615 hybrid_clock.cc:648] HybridClock initialized: now 1785982378729401 us; error 24 us; skew 500 ppm
Aug 06 02:12:58 dist-test-slave-6k22 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982378, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:58.732059 26615 init.cc:377] Logged in from keytab as oryx/127.18.70.65@KRBTEST.COM (short username oryx)
I20260806 02:12:58.733083 26615 webserver.cc:460] Webserver started at http://127.18.70.65:46003/ using document root <none> and password file <none>
I20260806 02:12:58.733613 26615 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:58.733672 26615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:58.733839 26615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:58.736177 26615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "979fdbf695d64161b91154eb270df219"
format_stamp: "Formatted at 2026-08-06 02:12:58 on dist-test-slave-6k22"
server_key: "2f83eee6f5d1bba06e81eaad68771e5f"
server_key_iv: "3f9304d879cf3a69898c6ca3f2e7aee6"
server_key_version: "encryptionkey@0"
I20260806 02:12:58.736730 26615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "979fdbf695d64161b91154eb270df219"
format_stamp: "Formatted at 2026-08-06 02:12:58 on dist-test-slave-6k22"
server_key: "2f83eee6f5d1bba06e81eaad68771e5f"
server_key_iv: "3f9304d879cf3a69898c6ca3f2e7aee6"
server_key_version: "encryptionkey@0"
I20260806 02:12:58.740684 26615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 02:12:58.743142 26631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:58.744309 26615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:58.744402 26615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "979fdbf695d64161b91154eb270df219"
format_stamp: "Formatted at 2026-08-06 02:12:58 on dist-test-slave-6k22"
server_key: "2f83eee6f5d1bba06e81eaad68771e5f"
server_key_iv: "3f9304d879cf3a69898c6ca3f2e7aee6"
server_key_version: "encryptionkey@0"
I20260806 02:12:58.744498 26615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:58.780431 26615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:58.783262 26615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:58.783464 26615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:58.784938 26615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:58.785012 26615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:58.785092 26615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:58.785148 26615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 02:12:58 dist-test-slave-6k22 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982378, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
I20260806 02:12:58.796247 26615 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:39461
I20260806 02:12:58.796316 26748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:39461 every 8 connection(s)
I20260806 02:12:58.797303 26615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:12:58.798527 26590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:58.785831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:35537 (local address 127.18.70.126:44033)
0806 02:12:58.786022 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:58.786028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:58.786798 (+ 770us) server_negotiation.cc:407] Connection header received
0806 02:12:58.787962 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:58.787966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:58.788021 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:58.788103 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:58.789669 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:58.790251 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:58.791594 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:58.791865 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:58.791988 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:58.794688 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:58.794694 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:58.794698 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:58.794730 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:58.796459 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:58.796960 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:58.796966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:58.796970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:58.797059 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:58.797367 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:58.797371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:58.797372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:58.797598 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:58.797714 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:58.798283 (+ 569us) server_negotiation.cc:299] Negotiation successful
0806 02:12:58.798409 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 02:12:58.799273 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:58.786182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44033 (local address 127.18.70.65:35537)
0806 02:12:58.786630 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:58.786678 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:58.787707 (+ 1029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:58.788512 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:58.788518 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:58.788854 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:58.789439 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:58.789456 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:58.790382 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:58.790386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:58.791443 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:58.791455 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:58.792049 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:58.792054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:58.792223 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:58.792799 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:58.792819 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:58.794522 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:58.796576 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:58.796580 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:58.796590 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:58.796856 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:58.797146 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:58.797147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:58.797149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:58.797271 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:58.797720 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:58.797726 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:58.798011 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 02:12:58.798201 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 02:12:58.798455 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":126,"threads_started":1}
I20260806 02:12:58.800629 26749 heartbeater.cc:347] Connected to a master server at 127.18.70.126:44033
I20260806 02:12:58.800806 26749 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:58.801005 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 26615
I20260806 02:12:58.801146 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:12:58.801402 26749 heartbeater.cc:511] Master 127.18.70.126:44033 requested a full tablet report, sending...
I20260806 02:12:58.801496 18713 external_mini_cluster.cc:1428] Setting key 05a9c4ccdffb918a44abc087425d3475
I20260806 02:12:58.803495 26534 ts_manager.cc:194] Registered new tserver with Master: 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461)
I20260806 02:12:58.804798 26534 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:35537
I20260806 02:12:58.810637 26756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:58.803001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44033 (local address 127.18.70.65:42151)
0806 02:12:58.803766 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:58.803793 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:58.803892 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:58.804129 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:58.804135 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:58.804375 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:58.804526 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:58.804536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:58.805486 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:58.805490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:58.806129 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:58.806138 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:58.806593 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:58.806596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:58.806688 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:58.807199 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:58.807210 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:58.807525 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:58.809168 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:58.809172 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:58.809174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:58.809369 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:58.809646 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:58.809650 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:58.809653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:58.809737 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:58.810103 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:58.810106 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:58.810202 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 02:12:58.810307 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 02:12:58.810444 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":157,"threads_started":1}
I20260806 02:12:58.810629 26590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:58.802969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:42151 (local address 127.18.70.126:44033)
0806 02:12:58.803080 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:58.803084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:58.803875 (+ 791us) server_negotiation.cc:407] Connection header received
0806 02:12:58.803971 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:58.803973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:58.804013 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:58.804094 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:58.804652 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:58.805345 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:58.806261 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:58.806481 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:58.806589 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:58.807688 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:58.807692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:58.807695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:58.807718 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:58.809055 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:58.809466 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:58.809470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:58.809471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:58.809556 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:58.809833 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:58.809837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:58.809839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:58.810008 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:58.810111 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:58.810361 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 02:12:58.810468 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
WARNING: no policy specified for oryx/127.18.70.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.70.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab.
Entry for principal oryx/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab.
I20260806 02:12:58.830755 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:44033
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab
--principal=oryx/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:58.946131 26760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:58.946476 26760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:58.946532 26760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:58.950750 26760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:58.950812 26760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:58.950853 26760 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 02:12:58.950888 26760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:12:58.955924 26760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab
--principal=oryx/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.70.126:44033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:58.956950 26760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:58.957922 26760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:58.967826 26768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:58.967842 26765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:58.967813 26766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:58.968109 26760 server_base.cc:1034] running on GCE node
I20260806 02:12:58.968576 26760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:58.969231 26760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:58.970428 26760 hybrid_clock.cc:648] HybridClock initialized: now 1785982378970392 us; error 53 us; skew 500 ppm
Aug 06 02:12:58 dist-test-slave-6k22 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982378, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:58.973469 26760 init.cc:377] Logged in from keytab as oryx/127.18.70.66@KRBTEST.COM (short username oryx)
I20260806 02:12:58.974653 26760 webserver.cc:460] Webserver started at http://127.18.70.66:42837/ using document root <none> and password file <none>
I20260806 02:12:58.975265 26760 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:58.975351 26760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:58.975652 26760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:58.977752 26760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "9b84105c8d244dcba15fd88d1c771563"
format_stamp: "Formatted at 2026-08-06 02:12:58 on dist-test-slave-6k22"
server_key: "f03edf8c01d4e502e95c47257c802c95"
server_key_iv: "b7626189a2ab411d502dfcfb3fc8e3ea"
server_key_version: "encryptionkey@0"
I20260806 02:12:58.978371 26760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "9b84105c8d244dcba15fd88d1c771563"
format_stamp: "Formatted at 2026-08-06 02:12:58 on dist-test-slave-6k22"
server_key: "f03edf8c01d4e502e95c47257c802c95"
server_key_iv: "b7626189a2ab411d502dfcfb3fc8e3ea"
server_key_version: "encryptionkey@0"
I20260806 02:12:58.982103 26760 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 02:12:58.984612 26775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:58.985846 26760 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:58.985970 26760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "9b84105c8d244dcba15fd88d1c771563"
format_stamp: "Formatted at 2026-08-06 02:12:58 on dist-test-slave-6k22"
server_key: "f03edf8c01d4e502e95c47257c802c95"
server_key_iv: "b7626189a2ab411d502dfcfb3fc8e3ea"
server_key_version: "encryptionkey@0"
I20260806 02:12:58.986073 26760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:59.009969 26760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:59.013018 26760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:59.013217 26760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:59.014940 26760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:59.014990 26760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:59.015050 26760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:59.015084 26760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 02:12:59 dist-test-slave-6k22 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982378, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
I20260806 02:12:59.026607 26760 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:34029
I20260806 02:12:59.026686 26895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:34029 every 8 connection(s)
I20260806 02:12:59.027657 26760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:12:59.029434 26784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.016220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44033 (local address 127.18.70.66:51161)
0806 02:12:59.016669 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:59.016720 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:59.017721 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:59.018625 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:59.018634 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:59.019072 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:59.019763 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.019786 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.020661 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.020664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.021545 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.021555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.022194 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.022199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.022360 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.022895 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:59.022919 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:59.024581 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:59.026704 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:59.026712 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:59.026727 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:59.027005 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:59.027316 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:59.027318 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:59.027319 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:59.027448 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:59.027926 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:59.027931 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:59.028223 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 02:12:59.028432 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 02:12:59.028722 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":131,"threads_started":1}
I20260806 02:12:59.030020 26590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.015806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:51161 (local address 127.18.70.126:44033)
0806 02:12:59.015980 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.015986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.016846 (+ 860us) server_negotiation.cc:407] Connection header received
0806 02:12:59.017999 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.018023 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.018086 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.018146 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:59.019916 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.020531 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.021670 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.022021 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.022141 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.024745 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:59.024748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:59.024751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:59.024777 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:59.026589 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:59.027113 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:59.027117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:59.027119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:59.027212 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:59.027549 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:59.027553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:59.027580 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:59.027820 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:59.027925 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.029804 (+ 1879us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.029911 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 02:12:59.031073 26896 heartbeater.cc:347] Connected to a master server at 127.18.70.126:44033
I20260806 02:12:59.031236 26896 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:59.031769 26896 heartbeater.cc:511] Master 127.18.70.126:44033 requested a full tablet report, sending...
I20260806 02:12:59.032774 26534 ts_manager.cc:194] Registered new tserver with Master: 9b84105c8d244dcba15fd88d1c771563 (127.18.70.66:34029)
I20260806 02:12:59.033586 26534 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:51161
I20260806 02:12:59.038005 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 26760
I20260806 02:12:59.038147 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:12:59.038501 18713 external_mini_cluster.cc:1428] Setting key da14f5a62bfecf28c3766d0f56aa06bf
I20260806 02:12:59.040649 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.033156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44033 (local address 127.18.70.66:36143)
0806 02:12:59.034019 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:59.034033 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:59.034108 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:59.034347 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:59.034350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:59.034545 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:59.034662 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.034670 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.035388 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.035391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.036078 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.036088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.036481 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.036484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.036551 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.036964 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:59.036975 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:59.037256 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:59.039110 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:59.039114 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:59.039116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:59.039378 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:59.039669 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:59.039674 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:59.039679 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:59.039774 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:59.040260 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:59.040263 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:59.040342 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 02:12:59.040428 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 02:12:59.040528 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"thread_start_us":107,"threads_started":1}
I20260806 02:12:59.040854 26590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.033248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:36143 (local address 127.18.70.126:44033)
0806 02:12:59.033366 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.033369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.034100 (+ 731us) server_negotiation.cc:407] Connection header received
0806 02:12:59.034170 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.034173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.034227 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.034313 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:59.034792 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.035260 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.036182 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.036387 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.036463 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.037376 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:59.037379 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:59.037381 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:59.037401 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:59.038999 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:59.039471 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:59.039474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:59.039475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:59.039548 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:59.039879 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:59.039884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:59.039885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:59.040146 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:59.040249 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.040517 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.040641 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for oryx/127.18.70.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.70.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab.
Entry for principal oryx/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab.
I20260806 02:12:59.067070 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:44033
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab
--principal=oryx/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:12:59.178572 26905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:12:59.178824 26905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:12:59.178906 26905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:12:59.182838 26905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:12:59.182917 26905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:12:59.182991 26905 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 02:12:59.183076 26905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:12:59.188110 26905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab
--principal=oryx/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.70.126:44033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:12:59.189111 26905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:12:59.190073 26905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:12:59.199427 26913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:12:59.199432 26911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:59.199685 26905 server_base.cc:1034] running on GCE node
W20260806 02:12:59.199883 26910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:12:59.200413 26905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:12:59.201036 26905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:12:59.202222 26905 hybrid_clock.cc:648] HybridClock initialized: now 1785982379202204 us; error 30 us; skew 500 ppm
Aug 06 02:12:59 dist-test-slave-6k22 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982379, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:12:59.204981 26905 init.cc:377] Logged in from keytab as oryx/127.18.70.67@KRBTEST.COM (short username oryx)
I20260806 02:12:59.205739 26905 webserver.cc:460] Webserver started at http://127.18.70.67:34389/ using document root <none> and password file <none>
I20260806 02:12:59.206225 26905 fs_manager.cc:359] Metadata directory not provided
I20260806 02:12:59.206288 26905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:12:59.206439 26905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:12:59.208357 26905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "52495f3b1eff44f9b97d82b68f6b88f4"
format_stamp: "Formatted at 2026-08-06 02:12:59 on dist-test-slave-6k22"
server_key: "403d12204ae9b26f926a07cf3187751a"
server_key_iv: "26085ffd589b5c77e6bf60ea363f8da3"
server_key_version: "encryptionkey@0"
I20260806 02:12:59.208844 26905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "52495f3b1eff44f9b97d82b68f6b88f4"
format_stamp: "Formatted at 2026-08-06 02:12:59 on dist-test-slave-6k22"
server_key: "403d12204ae9b26f926a07cf3187751a"
server_key_iv: "26085ffd589b5c77e6bf60ea363f8da3"
server_key_version: "encryptionkey@0"
I20260806 02:12:59.212328 26905 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:12:59.214960 26920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:59.216192 26905 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:12:59.216298 26905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "52495f3b1eff44f9b97d82b68f6b88f4"
format_stamp: "Formatted at 2026-08-06 02:12:59 on dist-test-slave-6k22"
server_key: "403d12204ae9b26f926a07cf3187751a"
server_key_iv: "26085ffd589b5c77e6bf60ea363f8da3"
server_key_version: "encryptionkey@0"
I20260806 02:12:59.216377 26905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:12:59.229278 26905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:12:59.232120 26905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:12:59.232256 26905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:12:59.233508 26905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:12:59.233582 26905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:12:59.233633 26905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:12:59.233660 26905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 02:12:59 dist-test-slave-6k22 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982379, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
I20260806 02:12:59.244596 26905 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:43521
I20260806 02:12:59.244732 27042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:43521 every 8 connection(s)
I20260806 02:12:59.245721 26905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:12:59.248308 26590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.234441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:48391 (local address 127.18.70.126:44033)
0806 02:12:59.234567 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.234571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.235709 (+ 1138us) server_negotiation.cc:407] Connection header received
0806 02:12:59.236733 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.236736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.236822 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.236889 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:59.238622 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.239523 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.241268 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.241620 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.241739 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.244683 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:59.244687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:59.244690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:59.244716 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:59.246308 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:59.246854 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:59.246857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:59.246858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:59.246937 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:59.247188 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:59.247191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:59.247192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:59.247377 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:59.247451 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.248018 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.248133 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 02:12:59.249018 26931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.234970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44033 (local address 127.18.70.67:48391)
0806 02:12:59.235457 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:59.235497 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:59.236480 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:59.237349 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:59.237358 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:59.237789 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:59.238443 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.238465 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.239716 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.239720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.241136 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.241149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.241773 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.241776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.241909 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.242561 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:59.242584 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:59.244506 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:59.246450 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:59.246458 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:59.246476 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:59.246736 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:59.247002 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:59.247004 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:59.247005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:59.247114 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:59.247467 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:59.247471 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:59.247781 (+ 310us) client_negotiation.cc:769] Sending connection context
0806 02:12:59.247968 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 02:12:59.248252 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":144,"threads_started":1}
I20260806 02:12:59.250089 27043 heartbeater.cc:347] Connected to a master server at 127.18.70.126:44033
I20260806 02:12:59.250224 27043 heartbeater.cc:464] Registering TS with master...
I20260806 02:12:59.250728 27043 heartbeater.cc:511] Master 127.18.70.126:44033 requested a full tablet report, sending...
I20260806 02:12:59.251827 26533 ts_manager.cc:194] Registered new tserver with Master: 52495f3b1eff44f9b97d82b68f6b88f4 (127.18.70.67:43521)
I20260806 02:12:59.252421 26533 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:48391
I20260806 02:12:59.253937 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 26905
I20260806 02:12:59.254079 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:12:59.254365 18713 external_mini_cluster.cc:1428] Setting key 6a17380a60c39845b8402de51bad5f30
I20260806 02:12:59.256479 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:12:59.259595 26590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.252462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:48727 (local address 127.18.70.126:44033)
0806 02:12:59.252565 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.252569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.252765 (+ 196us) server_negotiation.cc:407] Connection header received
0806 02:12:59.252896 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.252899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.252940 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.253012 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:59.253408 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.254091 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.255117 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.255333 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.255443 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.256470 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:59.256473 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:59.256476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:59.256498 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:59.257964 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:59.258400 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:59.258403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:59.258405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:59.258488 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:59.258782 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:59.258785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:59.258787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:59.258949 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:59.259053 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.259311 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.259415 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:12:59.259613 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.252398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44033 (local address 127.18.70.67:48727)
0806 02:12:59.252675 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:59.252686 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:59.252800 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:59.253042 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:59.253044 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:59.253194 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:59.253298 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.253303 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.254252 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.254255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.254985 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.254995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.255447 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.255451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.255542 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.256019 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:59.256029 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:59.256325 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:59.258079 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:59.258084 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:59.258087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:59.258284 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:59.258580 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:59.258584 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:59.258586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:59.258674 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:59.259048 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:59.259051 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:59.259144 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 02:12:59.259259 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 02:12:59.259391 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":102,"threads_started":1}
Aug 06 02:12:59 dist-test-slave-6k22 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982379, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 02:12:59 dist-test-slave-6k22 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:12:59 dist-test-slave-6k22 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 02:12:59.281100 26590 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:12:59.275397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53400 (local address 127.18.70.126:44033)
0806 02:12:59.275520 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.275524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.275536 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:12:59.275631 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.275633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.275670 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.275738 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:59.276415 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.276908 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.277681 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.277888 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.277961 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.280956 (+ 2995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53400: BlockingRecv error: recv got EOF from 127.0.0.1:53400 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 02:12:59 dist-test-slave-6k22 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982379, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
I20260806 02:12:59.289933 26590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.282350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53402 (local address 127.18.70.126:44033)
0806 02:12:59.282471 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.282476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.282569 (+ 93us) server_negotiation.cc:407] Connection header received
0806 02:12:59.282707 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.282709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.282752 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.282834 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:59.283371 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.283907 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.284721 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.284936 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.285039 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.287090 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:59.287093 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:59.287095 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:59.287117 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:59.288417 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:59.288863 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:59.288866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:59.288867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:59.288936 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:59.289231 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:59.289234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:59.289235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:59.289415 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:59.289514 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.289703 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.289787 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 02:12:59.292651 26533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:12:59.295615 26533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:59.310316 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.304508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:34029 (local address 127.0.0.1:41966)
0806 02:12:59.305563 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:59.305576 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:59.305668 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:59.306275 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:59.306278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:59.306327 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:59.306512 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.306517 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.307860 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.307863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.309000 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.309011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.309793 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.309797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.309908 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.309949 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 02:12:59.310061 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 02:12:59.310175 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":992,"mutex_wait_us":50,"spinlock_wait_cycles":53120,"thread_start_us":79,"threads_started":1}
I20260806 02:12:59.310702 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.304264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:39461 (local address 127.0.0.1:34906)
0806 02:12:59.305396 (+ 1132us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:59.305407 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:59.305484 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:59.305995 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:59.306001 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:59.306023 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:59.306154 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.306159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.307435 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.307441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.308927 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.308938 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.310146 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.310149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.310209 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.310225 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 02:12:59.310314 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 02:12:59.310589 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1076,"thread_start_us":95,"threads_started":1}
I20260806 02:12:59.311002 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.304897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41966 (local address 127.18.70.66:34029)
0806 02:12:59.305410 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.305418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.305680 (+ 262us) server_negotiation.cc:407] Connection header received
0806 02:12:59.305776 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.305781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.305978 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.306133 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:59.306634 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.307729 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.309100 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.309667 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.309792 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.309960 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.310602 (+ 642us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.310696 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":117,"threads_started":1}
I20260806 02:12:59.311208 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.304825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34906 (local address 127.18.70.65:39461)
0806 02:12:59.305240 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.305247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.305501 (+ 254us) server_negotiation.cc:407] Connection header received
0806 02:12:59.305588 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.305599 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.305844 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.306010 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:59.306266 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.307314 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.309068 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.309847 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.309946 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.310087 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.310350 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.310460 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":88,"threads_started":1}
I20260806 02:12:59.312070 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.304089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:43521 (local address 127.0.0.1:43504)
0806 02:12:59.304814 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:59.304861 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:59.304980 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:59.306275 (+ 1295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:59.306278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:59.306312 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:59.306509 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.306517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.308832 (+ 2315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.308838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.309954 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.309964 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.311665 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.311669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.311785 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.311808 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 02:12:59.311870 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 02:12:59.311960 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":38,"thread_start_us":55,"threads_started":1}
I20260806 02:12:59.312314 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.304260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43504 (local address 127.18.70.67:43521)
0806 02:12:59.305741 (+ 1481us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.305753 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.305773 (+ 20us) server_negotiation.cc:407] Connection header received
0806 02:12:59.305837 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.305842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.305997 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.306133 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:59.307889 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.308708 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.310077 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.310847 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.310953 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.312024 (+ 1071us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.312082 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.312170 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1351,"thread_start_us":108,"threads_started":1}
I20260806 02:12:59.313874 26828 tablet_service.cc:1467] Processing CreateTablet for tablet 02c9802288724eedb72761fa29f8442a (DEFAULT_TABLE table=test-table [id=229bbc7bdd664922a31846edbf41c49f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:59.313903 26682 tablet_service.cc:1467] Processing CreateTablet for tablet 02c9802288724eedb72761fa29f8442a (DEFAULT_TABLE table=test-table [id=229bbc7bdd664922a31846edbf41c49f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:59.314921 26828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02c9802288724eedb72761fa29f8442a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:59.314932 26682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02c9802288724eedb72761fa29f8442a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:59.317481 26976 tablet_service.cc:1467] Processing CreateTablet for tablet 02c9802288724eedb72761fa29f8442a (DEFAULT_TABLE table=test-table [id=229bbc7bdd664922a31846edbf41c49f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:59.318586 26976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02c9802288724eedb72761fa29f8442a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:59.322367 27072 tablet_bootstrap.cc:492] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563: Bootstrap starting.
I20260806 02:12:59.322968 27071 tablet_bootstrap.cc:492] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219: Bootstrap starting.
I20260806 02:12:59.324461 27072 tablet_bootstrap.cc:654] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:59.325178 27071 tablet_bootstrap.cc:654] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:59.325205 27072 log.cc:824] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:59.326010 27071 log.cc:824] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:59.326699 27074 tablet_bootstrap.cc:492] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4: Bootstrap starting.
I20260806 02:12:59.327405 27072 tablet_bootstrap.cc:492] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563: No bootstrap required, opened a new log
I20260806 02:12:59.327652 27072 ts_tablet_manager.cc:1397] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 02:12:59.327903 27071 tablet_bootstrap.cc:492] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219: No bootstrap required, opened a new log
I20260806 02:12:59.328068 27071 ts_tablet_manager.cc:1397] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 02:12:59.328739 27074 tablet_bootstrap.cc:654] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:59.329514 27074 log.cc:824] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4: Log is configured to *not* fsync() on all Append() calls
I20260806 02:12:59.331125 27072 raft_consensus.cc:348] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } }
I20260806 02:12:59.331355 27072 raft_consensus.cc:374] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:59.331411 27072 raft_consensus.cc:729] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b84105c8d244dcba15fd88d1c771563, State: Initialized, Role: FOLLOWER
I20260806 02:12:59.331524 27074 tablet_bootstrap.cc:492] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4: No bootstrap required, opened a new log
I20260806 02:12:59.331734 27074 ts_tablet_manager.cc:1397] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260806 02:12:59.331930 27072 consensus_queue.cc:260] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } }
I20260806 02:12:59.332149 27071 raft_consensus.cc:348] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } }
I20260806 02:12:59.332393 27071 raft_consensus.cc:374] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:59.332443 27071 raft_consensus.cc:729] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979fdbf695d64161b91154eb270df219, State: Initialized, Role: FOLLOWER
I20260806 02:12:59.332952 27072 ts_tablet_manager.cc:1428] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 02:12:59.332885 27071 consensus_queue.cc:260] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } }
I20260806 02:12:59.333328 26896 heartbeater.cc:503] Master 127.18.70.126:44033 was elected leader, sending a full tablet report...
I20260806 02:12:59.333456 26749 heartbeater.cc:503] Master 127.18.70.126:44033 was elected leader, sending a full tablet report...
I20260806 02:12:59.333797 27071 ts_tablet_manager.cc:1428] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:12:59.336614 27074 raft_consensus.cc:348] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } }
I20260806 02:12:59.336894 27074 raft_consensus.cc:374] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:59.336958 27074 raft_consensus.cc:729] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52495f3b1eff44f9b97d82b68f6b88f4, State: Initialized, Role: FOLLOWER
I20260806 02:12:59.337388 27074 consensus_queue.cc:260] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } }
I20260806 02:12:59.338267 27074 ts_tablet_manager.cc:1428] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260806 02:12:59.338290 27043 heartbeater.cc:503] Master 127.18.70.126:44033 was elected leader, sending a full tablet report...
W20260806 02:12:59.496796 27044 tablet.cc:2369] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 02:12:59.529049 26897 tablet.cc:2369] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 02:12:59.548830 26750 tablet.cc:2369] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:12:59.701500 27079 raft_consensus.cc:482] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:59.701764 27079 raft_consensus.cc:504] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } }
I20260806 02:12:59.703078 27079 leader_election.cc:288] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461), 9b84105c8d244dcba15fd88d1c771563 (127.18.70.66:34029)
I20260806 02:12:59.707580 26931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.703424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:34029 (local address 127.18.70.67:60819)
0806 02:12:59.703679 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:59.703695 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:59.703802 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:59.704118 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:59.704121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:59.704141 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:59.704278 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.704289 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.705321 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.705325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.706297 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.706306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.707121 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.707124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.707228 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.707255 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 02:12:59.707336 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 02:12:59.707406 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":1}
I20260806 02:12:59.707780 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.703611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:60819 (local address 127.18.70.66:34029)
0806 02:12:59.703744 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.703748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.703767 (+ 19us) server_negotiation.cc:407] Connection header received
0806 02:12:59.703931 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.703935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.704000 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.704087 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:59.704401 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.705216 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.706424 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.707013 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.707103 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.707148 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.707481 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.707615 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 02:12:59.708290 27080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.703424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:39461 (local address 127.18.70.67:49001)
0806 02:12:59.703950 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:59.703964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:59.704082 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:59.704305 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:59.704307 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:59.704321 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:59.704908 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.704919 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.705819 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.705822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.706790 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.706800 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.707872 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.707878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.707991 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.708014 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 02:12:59.708106 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 02:12:59.708169 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":57,"thread_start_us":146,"threads_started":1}
I20260806 02:12:59.708345 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.703523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:49001 (local address 127.18.70.65:39461)
0806 02:12:59.703707 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.703711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.704094 (+ 383us) server_negotiation.cc:407] Connection header received
0806 02:12:59.704157 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.704159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.704204 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.704296 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:59.705029 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.705703 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.706918 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.707482 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.707641 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.707690 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.708143 (+ 453us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.708222 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 02:12:59.708426 26849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02c9802288724eedb72761fa29f8442a" candidate_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b84105c8d244dcba15fd88d1c771563" is_pre_election: true
I20260806 02:12:59.708812 26849 raft_consensus.cc:2454] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52495f3b1eff44f9b97d82b68f6b88f4 in term 0.
I20260806 02:12:59.709096 26703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02c9802288724eedb72761fa29f8442a" candidate_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979fdbf695d64161b91154eb270df219" is_pre_election: true
I20260806 02:12:59.709380 26923 leader_election.cc:302] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52495f3b1eff44f9b97d82b68f6b88f4, 9b84105c8d244dcba15fd88d1c771563; no voters:
I20260806 02:12:59.709405 26703 raft_consensus.cc:2454] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52495f3b1eff44f9b97d82b68f6b88f4 in term 0.
I20260806 02:12:59.709638 27079 raft_consensus.cc:2781] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:59.709753 27079 raft_consensus.cc:482] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:59.709813 27079 raft_consensus.cc:3037] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:59.710861 27079 raft_consensus.cc:504] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } }
I20260806 02:12:59.711216 27079 leader_election.cc:288] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [CANDIDATE]: Term 1 election: Requested vote from peers 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461), 9b84105c8d244dcba15fd88d1c771563 (127.18.70.66:34029)
I20260806 02:12:59.711524 26703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02c9802288724eedb72761fa29f8442a" candidate_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979fdbf695d64161b91154eb270df219"
I20260806 02:12:59.711597 26849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02c9802288724eedb72761fa29f8442a" candidate_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b84105c8d244dcba15fd88d1c771563"
I20260806 02:12:59.711678 26703 raft_consensus.cc:3037] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:59.711745 26849 raft_consensus.cc:3037] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:59.712826 26849 raft_consensus.cc:2454] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52495f3b1eff44f9b97d82b68f6b88f4 in term 1.
I20260806 02:12:59.712909 26703 raft_consensus.cc:2454] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52495f3b1eff44f9b97d82b68f6b88f4 in term 1.
I20260806 02:12:59.713133 26923 leader_election.cc:302] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52495f3b1eff44f9b97d82b68f6b88f4, 9b84105c8d244dcba15fd88d1c771563; no voters:
I20260806 02:12:59.713326 27079 raft_consensus.cc:2781] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:59.713577 27079 raft_consensus.cc:686] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 LEADER]: Becoming Leader. State: Replica: 52495f3b1eff44f9b97d82b68f6b88f4, State: Running, Role: LEADER
I20260806 02:12:59.713949 27079 consensus_queue.cc:237] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } }
I20260806 02:12:59.717123 26534 catalog_manager.cc:5495] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52495f3b1eff44f9b97d82b68f6b88f4 (127.18.70.67). New cstate: current_term: 1 leader_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } health_report { overall_health: UNKNOWN } } }
I20260806 02:12:59.767201 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.763822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43506 (local address 127.18.70.67:43521)
0806 02:12:59.763960 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.763963 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.763978 (+ 15us) server_negotiation.cc:407] Connection header received
0806 02:12:59.764115 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.764117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.764174 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.764269 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:12:59.764487 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.765065 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.765888 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.766105 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.766184 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.766413 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:12:59.766840 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:12:59.766925 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.766991 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.767036 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:12:59.772958 26849 raft_consensus.cc:1261] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Refusing update from remote peer 52495f3b1eff44f9b97d82b68f6b88f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:59.772960 26703 raft_consensus.cc:1261] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Refusing update from remote peer 52495f3b1eff44f9b97d82b68f6b88f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:12:59.773629 27079 consensus_queue.cc:1035] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:59.773849 27081 consensus_queue.cc:1035] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:12:59.782330 27086 mvcc.cc:205] Tried to move back new op lower bound from 7315383827542736896 to 7315383827311190016. Current Snapshot: MvccSnapshot[applied={T|T < 7315383827542736896}]
I20260806 02:12:59.790413 26533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:59.790544 26533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:12:59.792833 26533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040: Sending DeleteTablet for 3 replicas of tablet 02c9802288724eedb72761fa29f8442a
I20260806 02:12:59.793525 26976 tablet_service.cc:1514] Processing DeleteTablet for tablet 02c9802288724eedb72761fa29f8442a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:59 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:43504
I20260806 02:12:59.793526 26682 tablet_service.cc:1514] Processing DeleteTablet for tablet 02c9802288724eedb72761fa29f8442a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:59 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:34906
I20260806 02:12:59.794186 27096 tablet_replica.cc:330] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219: stopping tablet replica
I20260806 02:12:59.794157 26828 tablet_service.cc:1514] Processing DeleteTablet for tablet 02c9802288724eedb72761fa29f8442a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:12:59 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:41966
I20260806 02:12:59.794443 26533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:12:59.794531 27096 raft_consensus.cc:2229] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 02:12:59.794808 26533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:59.794811 27096 raft_consensus.cc:2258] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:59.795205 27095 tablet_replica.cc:330] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4: stopping tablet replica
I20260806 02:12:59.795478 27095 raft_consensus.cc:2229] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:12:59.795620 27097 tablet_replica.cc:330] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563: stopping tablet replica
I20260806 02:12:59.795872 27095 raft_consensus.cc:2258] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:59.795859 27097 raft_consensus.cc:2229] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:12:59.796120 27097 raft_consensus.cc:2258] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:12:59.797356 27095 ts_tablet_manager.cc:1905] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:59.797855 27097 ts_tablet_manager.cc:1905] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:59.800393 27095 ts_tablet_manager.cc:1918] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:59.800484 27095 log.cc:1196] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/02c9802288724eedb72761fa29f8442a
I20260806 02:12:59.800719 27097 ts_tablet_manager.cc:1918] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:59.800809 27097 log.cc:1196] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/02c9802288724eedb72761fa29f8442a
I20260806 02:12:59.800884 27095 ts_tablet_manager.cc:1939] T 02c9802288724eedb72761fa29f8442a P 52495f3b1eff44f9b97d82b68f6b88f4: Deleting consensus metadata
I20260806 02:12:59.801164 27097 ts_tablet_manager.cc:1939] T 02c9802288724eedb72761fa29f8442a P 9b84105c8d244dcba15fd88d1c771563: Deleting consensus metadata
I20260806 02:12:59.802166 26521 catalog_manager.cc:4841] TS 52495f3b1eff44f9b97d82b68f6b88f4 (127.18.70.67:43521): tablet 02c9802288724eedb72761fa29f8442a (table test-table [id=229bbc7bdd664922a31846edbf41c49f]) successfully deleted
I20260806 02:12:59.802166 26520 catalog_manager.cc:4841] TS 9b84105c8d244dcba15fd88d1c771563 (127.18.70.66:34029): tablet 02c9802288724eedb72761fa29f8442a (table test-table [id=229bbc7bdd664922a31846edbf41c49f]) successfully deleted
I20260806 02:12:59.803491 27096 ts_tablet_manager.cc:1905] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:12:59.806208 26976 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdf1c6bb5c24f2ebd9f03e369710404 (DEFAULT_TABLE table=test-table [id=669fd5b43d1a4a4fbda07791f8a2575d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:59.805646 26682 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdf1c6bb5c24f2ebd9f03e369710404 (DEFAULT_TABLE table=test-table [id=669fd5b43d1a4a4fbda07791f8a2575d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:59.806463 26976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdf1c6bb5c24f2ebd9f03e369710404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:59.806721 26682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdf1c6bb5c24f2ebd9f03e369710404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:59.808832 27074 tablet_bootstrap.cc:492] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4: Bootstrap starting.
I20260806 02:12:59.809543 27096 ts_tablet_manager.cc:1918] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:12:59.809912 27074 tablet_bootstrap.cc:654] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:59.811591 27074 tablet_bootstrap.cc:492] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4: No bootstrap required, opened a new log
I20260806 02:12:59.811755 27074 ts_tablet_manager.cc:1397] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260806 02:12:59.811700 26828 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdf1c6bb5c24f2ebd9f03e369710404 (DEFAULT_TABLE table=test-table [id=669fd5b43d1a4a4fbda07791f8a2575d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:12:59.811888 27096 log.cc:1196] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/wal/wals/02c9802288724eedb72761fa29f8442a
I20260806 02:12:59.812016 26828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdf1c6bb5c24f2ebd9f03e369710404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:59.812376 27096 ts_tablet_manager.cc:1939] T 02c9802288724eedb72761fa29f8442a P 979fdbf695d64161b91154eb270df219: Deleting consensus metadata
I20260806 02:12:59.812317 27074 raft_consensus.cc:348] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.812448 27074 raft_consensus.cc:374] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:59.812505 27074 raft_consensus.cc:729] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52495f3b1eff44f9b97d82b68f6b88f4, State: Initialized, Role: FOLLOWER
I20260806 02:12:59.810459 27071 tablet_bootstrap.cc:492] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219: Bootstrap starting.
I20260806 02:12:59.813680 27071 tablet_bootstrap.cc:654] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:59.814395 27074 consensus_queue.cc:260] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.814627 27072 tablet_bootstrap.cc:492] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563: Bootstrap starting.
I20260806 02:12:59.814735 27074 ts_tablet_manager.cc:1428] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 02:12:59.815666 27072 tablet_bootstrap.cc:654] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:59.815750 27071 tablet_bootstrap.cc:492] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219: No bootstrap required, opened a new log
I20260806 02:12:59.815838 27071 ts_tablet_manager.cc:1397] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 02:12:59.816226 27071 raft_consensus.cc:348] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.816308 27071 raft_consensus.cc:374] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:59.816344 27071 raft_consensus.cc:729] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979fdbf695d64161b91154eb270df219, State: Initialized, Role: FOLLOWER
I20260806 02:12:59.816481 27071 consensus_queue.cc:260] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.816802 27071 ts_tablet_manager.cc:1428] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 02:12:59.818007 26519 catalog_manager.cc:4841] TS 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461): tablet 02c9802288724eedb72761fa29f8442a (table test-table [id=229bbc7bdd664922a31846edbf41c49f]) successfully deleted
I20260806 02:12:59.820528 27072 tablet_bootstrap.cc:492] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563: No bootstrap required, opened a new log
I20260806 02:12:59.820657 27072 ts_tablet_manager.cc:1397] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260806 02:12:59.821161 27072 raft_consensus.cc:348] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.821274 27072 raft_consensus.cc:374] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:59.821305 27072 raft_consensus.cc:729] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b84105c8d244dcba15fd88d1c771563, State: Initialized, Role: FOLLOWER
I20260806 02:12:59.821408 27072 consensus_queue.cc:260] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.821689 27072 ts_tablet_manager.cc:1428] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 02:12:59.911255 27077 raft_consensus.cc:482] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:12:59.911408 27077 raft_consensus.cc:504] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.912539 27077 leader_election.cc:288] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461), 52495f3b1eff44f9b97d82b68f6b88f4 (127.18.70.67:43521)
I20260806 02:12:59.917050 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.912911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:47071 (local address 127.18.70.67:43521)
0806 02:12:59.913053 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.913059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.913182 (+ 123us) server_negotiation.cc:407] Connection header received
0806 02:12:59.913488 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.913493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.913555 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.913674 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:59.913981 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.914748 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.916020 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.916452 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.916569 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.916624 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.916854 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.916922 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 02:12:59.917085 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.912811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:43521 (local address 127.18.70.66:47071)
0806 02:12:59.913089 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:59.913105 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:59.913266 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:59.913699 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:59.913704 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:59.913725 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:59.913872 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.913880 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.914873 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.914876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.915893 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.915903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.916586 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.916591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.916699 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.916732 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 02:12:59.916827 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 02:12:59.916903 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":46,"thread_start_us":87,"threads_started":1}
I20260806 02:12:59.917799 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.913135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:43565 (local address 127.18.70.65:39461)
0806 02:12:59.913265 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.913269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.913565 (+ 296us) server_negotiation.cc:407] Connection header received
0806 02:12:59.913752 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.913755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.913814 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.913868 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:12:59.914229 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.914875 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.916497 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.917005 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.917122 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.917172 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.917595 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.917672 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 02:12:59.917745 26996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdf1c6bb5c24f2ebd9f03e369710404" candidate_uuid: "9b84105c8d244dcba15fd88d1c771563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" is_pre_election: true
I20260806 02:12:59.917997 26996 raft_consensus.cc:2454] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b84105c8d244dcba15fd88d1c771563 in term 0.
I20260806 02:12:59.918262 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.912813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:39461 (local address 127.18.70.66:43565)
0806 02:12:59.913471 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:59.913490 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:59.913602 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:59.914007 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:59.914010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:59.914031 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:12:59.914119 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.914123 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.915015 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.915020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.916332 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.916357 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.917353 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.917357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.917458 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.917479 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 02:12:59.918006 (+ 527us) client_negotiation.cc:240] Negotiation successful
0806 02:12:59.918080 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":265,"thread_start_us":115,"threads_started":1}
I20260806 02:12:59.918538 26779 leader_election.cc:302] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52495f3b1eff44f9b97d82b68f6b88f4, 9b84105c8d244dcba15fd88d1c771563; no voters:
I20260806 02:12:59.918820 27077 raft_consensus.cc:2781] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:12:59.918895 27077 raft_consensus.cc:482] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:12:59.918946 27077 raft_consensus.cc:3037] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:59.919219 26703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdf1c6bb5c24f2ebd9f03e369710404" candidate_uuid: "9b84105c8d244dcba15fd88d1c771563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979fdbf695d64161b91154eb270df219" is_pre_election: true
I20260806 02:12:59.919366 26703 raft_consensus.cc:2454] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b84105c8d244dcba15fd88d1c771563 in term 0.
I20260806 02:12:59.920385 27077 raft_consensus.cc:504] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.920790 27077 leader_election.cc:288] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [CANDIDATE]: Term 1 election: Requested vote from peers 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461), 52495f3b1eff44f9b97d82b68f6b88f4 (127.18.70.67:43521)
I20260806 02:12:59.921056 26703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdf1c6bb5c24f2ebd9f03e369710404" candidate_uuid: "9b84105c8d244dcba15fd88d1c771563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979fdbf695d64161b91154eb270df219"
I20260806 02:12:59.921114 26996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdf1c6bb5c24f2ebd9f03e369710404" candidate_uuid: "9b84105c8d244dcba15fd88d1c771563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52495f3b1eff44f9b97d82b68f6b88f4"
I20260806 02:12:59.921201 26996 raft_consensus.cc:3037] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:59.921195 26703 raft_consensus.cc:3037] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:12:59.922293 26996 raft_consensus.cc:2454] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b84105c8d244dcba15fd88d1c771563 in term 1.
I20260806 02:12:59.922289 26703 raft_consensus.cc:2454] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b84105c8d244dcba15fd88d1c771563 in term 1.
I20260806 02:12:59.922580 26777 leader_election.cc:302] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 979fdbf695d64161b91154eb270df219, 9b84105c8d244dcba15fd88d1c771563; no voters:
I20260806 02:12:59.922806 27077 raft_consensus.cc:2781] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:12:59.923030 27077 raft_consensus.cc:686] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [term 1 LEADER]: Becoming Leader. State: Replica: 9b84105c8d244dcba15fd88d1c771563, State: Running, Role: LEADER
I20260806 02:12:59.923336 27077 consensus_queue.cc:237] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.925596 26534 catalog_manager.cc:5495] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b84105c8d244dcba15fd88d1c771563 (127.18.70.66). New cstate: current_term: 1 leader_uuid: "9b84105c8d244dcba15fd88d1c771563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } health_report { overall_health: UNKNOWN } } }
Aug 06 02:12:59 dist-test-slave-6k22 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982378, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.126@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
I20260806 02:12:59.962059 27111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.953241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44033 (local address 127.0.0.1:53410)
0806 02:12:59.953518 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 02:12:59.953535 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:12:59.953634 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:12:59.953933 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:12:59.953936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:12:59.954317 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:12:59.954417 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.954423 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.955657 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.955660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.956197 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:12:59.956204 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.956865 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.956867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:12:59.956957 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.957528 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:12:59.957540 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:12:59.959073 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:12:59.960730 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:59.960738 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:59.960742 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:59.960955 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:59.961202 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:12:59.961203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:12:59.961205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:12:59.961287 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:12:59.961623 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:12:59.961628 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:12:59.961720 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 02:12:59.961806 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 02:12:59.961926 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":17,"thread_start_us":112,"threads_started":1}
I20260806 02:12:59.962230 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:12:59.953325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.18.70.126:44033)
0806 02:12:59.953602 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 02:12:59.953611 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 02:12:59.953632 (+ 21us) server_negotiation.cc:407] Connection header received
0806 02:12:59.953738 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:12:59.953741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:12:59.953791 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:12:59.953892 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:12:59.954536 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.955507 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.956338 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:12:59.956741 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:12:59.956851 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:12:59.959215 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:12:59.959219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:12:59.959222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:12:59.959246 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:12:59.960634 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:59.961038 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:59.961041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:59.961042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:59.961115 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:12:59.961368 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:12:59.961371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:12:59.961372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:12:59.961535 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:12:59.961611 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 02:12:59.961928 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 02:12:59.962030 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":210,"thread_start_us":109,"threads_started":1}
I20260806 02:12:59.966054 26534 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:53410:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 02:12:59.966805 26534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:12:59.971437 26682 tablet_service.cc:1467] Processing CreateTablet for tablet 2d95ef874576480c955e6529baa9e936 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66c467fea48c4387900c6ee3fd5af46d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 02:12:59.971655 26828 tablet_service.cc:1467] Processing CreateTablet for tablet 2d95ef874576480c955e6529baa9e936 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66c467fea48c4387900c6ee3fd5af46d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 02:12:59.971745 26976 tablet_service.cc:1467] Processing CreateTablet for tablet 2d95ef874576480c955e6529baa9e936 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=66c467fea48c4387900c6ee3fd5af46d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 02:12:59.971928 26682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d95ef874576480c955e6529baa9e936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:59.972051 26828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d95ef874576480c955e6529baa9e936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:59.972093 26976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d95ef874576480c955e6529baa9e936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:12:59.974553 27072 tablet_bootstrap.cc:492] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563: Bootstrap starting.
I20260806 02:12:59.974689 27074 tablet_bootstrap.cc:492] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4: Bootstrap starting.
I20260806 02:12:59.975395 27074 tablet_bootstrap.cc:654] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:59.975453 27072 tablet_bootstrap.cc:654] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:59.976066 27071 tablet_bootstrap.cc:492] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219: Bootstrap starting.
I20260806 02:12:59.977008 27071 tablet_bootstrap.cc:654] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219: Neither blocks nor log segments found. Creating new log.
I20260806 02:12:59.977618 27072 tablet_bootstrap.cc:492] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563: No bootstrap required, opened a new log
I20260806 02:12:59.977722 27072 ts_tablet_manager.cc:1397] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 02:12:59.978178 27072 raft_consensus.cc:348] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.978318 27072 raft_consensus.cc:374] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:59.978332 27074 tablet_bootstrap.cc:492] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4: No bootstrap required, opened a new log
I20260806 02:12:59.978351 27072 raft_consensus.cc:729] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b84105c8d244dcba15fd88d1c771563, State: Initialized, Role: FOLLOWER
I20260806 02:12:59.978438 27074 ts_tablet_manager.cc:1397] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 02:12:59.978535 27072 consensus_queue.cc:260] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.978839 27077 tablet_replica.cc:439] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } }
I20260806 02:12:59.978878 27074 raft_consensus.cc:348] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.978968 27077 tablet_replica.cc:442] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:59.978988 27074 raft_consensus.cc:374] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:59.979032 27074 raft_consensus.cc:729] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52495f3b1eff44f9b97d82b68f6b88f4, State: Initialized, Role: FOLLOWER
I20260806 02:12:59.979156 27074 consensus_queue.cc:260] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.979260 27072 ts_tablet_manager.cc:1428] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 02:12:59.979408 27074 ts_tablet_manager.cc:1428] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:12:59.979805 27090 tablet_replica.cc:439] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } }
I20260806 02:12:59.979940 27090 tablet_replica.cc:442] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:12:59.980229 27071 tablet_bootstrap.cc:492] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219: No bootstrap required, opened a new log
I20260806 02:12:59.980294 27071 ts_tablet_manager.cc:1397] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 02:12:59.980706 27071 raft_consensus.cc:348] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.980849 27071 raft_consensus.cc:374] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:12:59.980904 27071 raft_consensus.cc:729] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979fdbf695d64161b91154eb270df219, State: Initialized, Role: FOLLOWER
I20260806 02:12:59.981030 27071 consensus_queue.cc:260] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:12:59.981331 27071 ts_tablet_manager.cc:1428] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 02:12:59.981293 27078 tablet_replica.cc:439] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } }
I20260806 02:12:59.981428 27078 tablet_replica.cc:442] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:13:00.015136 27077 raft_consensus.cc:482] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:13:00.015295 27077 raft_consensus.cc:504] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.015785 27077 leader_election.cc:288] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461), 52495f3b1eff44f9b97d82b68f6b88f4 (127.18.70.67:43521)
I20260806 02:13:00.016222 26703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d95ef874576480c955e6529baa9e936" candidate_uuid: "9b84105c8d244dcba15fd88d1c771563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979fdbf695d64161b91154eb270df219" is_pre_election: true
I20260806 02:13:00.016263 26996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d95ef874576480c955e6529baa9e936" candidate_uuid: "9b84105c8d244dcba15fd88d1c771563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" is_pre_election: true
I20260806 02:13:00.016386 26703 raft_consensus.cc:2454] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b84105c8d244dcba15fd88d1c771563 in term 0.
I20260806 02:13:00.016503 26996 raft_consensus.cc:2454] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b84105c8d244dcba15fd88d1c771563 in term 0.
I20260806 02:13:00.016729 26777 leader_election.cc:302] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 979fdbf695d64161b91154eb270df219, 9b84105c8d244dcba15fd88d1c771563; no voters:
I20260806 02:13:00.016894 27077 raft_consensus.cc:2781] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:13:00.016955 27077 raft_consensus.cc:482] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:13:00.016990 27077 raft_consensus.cc:3037] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:00.017992 27077 raft_consensus.cc:504] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.018317 27077 leader_election.cc:288] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [CANDIDATE]: Term 1 election: Requested vote from peers 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461), 52495f3b1eff44f9b97d82b68f6b88f4 (127.18.70.67:43521)
I20260806 02:13:00.018697 26703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d95ef874576480c955e6529baa9e936" candidate_uuid: "9b84105c8d244dcba15fd88d1c771563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979fdbf695d64161b91154eb270df219"
I20260806 02:13:00.018853 26703 raft_consensus.cc:3037] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:00.018860 26996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d95ef874576480c955e6529baa9e936" candidate_uuid: "9b84105c8d244dcba15fd88d1c771563" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52495f3b1eff44f9b97d82b68f6b88f4"
I20260806 02:13:00.019016 26996 raft_consensus.cc:3037] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:00.019901 26703 raft_consensus.cc:2454] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b84105c8d244dcba15fd88d1c771563 in term 1.
I20260806 02:13:00.020138 26996 raft_consensus.cc:2454] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b84105c8d244dcba15fd88d1c771563 in term 1.
I20260806 02:13:00.020143 26777 leader_election.cc:302] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 979fdbf695d64161b91154eb270df219, 9b84105c8d244dcba15fd88d1c771563; no voters:
I20260806 02:13:00.020341 27077 raft_consensus.cc:2781] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:13:00.020443 27077 raft_consensus.cc:686] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [term 1 LEADER]: Becoming Leader. State: Replica: 9b84105c8d244dcba15fd88d1c771563, State: Running, Role: LEADER
I20260806 02:13:00.020633 27077 consensus_queue.cc:237] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.021363 27103 tablet_replica.cc:439] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563: TxnStatusTablet state changed. Reason: New leader 9b84105c8d244dcba15fd88d1c771563. Latest consensus state: current_term: 1 leader_uuid: "9b84105c8d244dcba15fd88d1c771563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } }
I20260806 02:13:00.021483 27103 tablet_replica.cc:442] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563: This TxnStatusTablet replica's current role is: LEADER
I20260806 02:13:00.021802 27117 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 02:13:00.021989 27117 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 02:13:00.022451 26533 catalog_manager.cc:5495] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b84105c8d244dcba15fd88d1c771563 (127.18.70.66). New cstate: current_term: 1 leader_uuid: "9b84105c8d244dcba15fd88d1c771563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } health_report { overall_health: UNKNOWN } } }
I20260806 02:13:00.045037 27111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:00.040938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:34029 (local address 127.0.0.1:41972)
0806 02:13:00.041161 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:00.041176 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:00.041276 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:00.041604 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:00.041607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:00.041615 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 02:13:00.041720 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:00.041726 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.042718 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.042721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:00.043431 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:00.043440 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.043981 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.043983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:00.044043 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:00.044085 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 02:13:00.044796 (+ 711us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 02:13:00.044807 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 02:13:00.044865 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 02:13:00.044932 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260806 02:13:00.045056 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:00.041040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41972 (local address 127.18.70.66:34029)
0806 02:13:00.041148 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:00.041152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:00.041239 (+ 87us) server_negotiation.cc:407] Connection header received
0806 02:13:00.041409 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:00.041412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:00.041473 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:00.041574 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:13:00.041828 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.042585 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.043541 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.043875 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.043975 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:00.044212 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:13:00.044704 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:13:00.044816 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:00.044880 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 02:13:00.044932 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 02:13:00.047739 26703 raft_consensus.cc:1261] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Refusing update from remote peer 9b84105c8d244dcba15fd88d1c771563: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:13:00.047909 26996 raft_consensus.cc:1261] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Refusing update from remote peer 9b84105c8d244dcba15fd88d1c771563: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:13:00.048236 27077 consensus_queue.cc:1035] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:13:00.048658 27077 consensus_queue.cc:1035] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:13:00.049736 27078 tablet_replica.cc:439] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219: TxnStatusTablet state changed. Reason: New leader 9b84105c8d244dcba15fd88d1c771563. Latest consensus state: current_term: 1 leader_uuid: "9b84105c8d244dcba15fd88d1c771563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } }
I20260806 02:13:00.049858 27078 tablet_replica.cc:442] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:13:00.051124 27090 tablet_replica.cc:439] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4: TxnStatusTablet state changed. Reason: New leader 9b84105c8d244dcba15fd88d1c771563. Latest consensus state: current_term: 1 leader_uuid: "9b84105c8d244dcba15fd88d1c771563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } }
I20260806 02:13:00.051250 27090 tablet_replica.cc:442] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:13:00.053261 27087 mvcc.cc:205] Tried to move back new op lower bound from 7315383828670476288 to 7315383828566052864. Current Snapshot: MvccSnapshot[applied={T|T < 7315383828670476288}]
I20260806 02:13:00.053427 27078 tablet_replica.cc:439] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b84105c8d244dcba15fd88d1c771563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } }
I20260806 02:13:00.053520 27078 tablet_replica.cc:442] T 2d95ef874576480c955e6529baa9e936 P 979fdbf695d64161b91154eb270df219: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:13:00.053635 27084 mvcc.cc:205] Tried to move back new op lower bound from 7315383828670476288 to 7315383828566052864. Current Snapshot: MvccSnapshot[applied={T|T < 7315383828670476288}]
I20260806 02:13:00.053797 27090 tablet_replica.cc:439] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b84105c8d244dcba15fd88d1c771563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } }
I20260806 02:13:00.053894 27090 tablet_replica.cc:442] T 2d95ef874576480c955e6529baa9e936 P 52495f3b1eff44f9b97d82b68f6b88f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 02:13:00.055339 27077 tablet_replica.cc:439] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b84105c8d244dcba15fd88d1c771563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } }
I20260806 02:13:00.055446 27077 tablet_replica.cc:442] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563: This TxnStatusTablet replica's current role is: LEADER
I20260806 02:13:00.055420 27103 tablet_replica.cc:439] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b84105c8d244dcba15fd88d1c771563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } } }
I20260806 02:13:00.055516 27103 tablet_replica.cc:442] T 2d95ef874576480c955e6529baa9e936 P 9b84105c8d244dcba15fd88d1c771563: This TxnStatusTablet replica's current role is: LEADER
I20260806 02:13:00.061075 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:00.057784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41982 (local address 127.18.70.66:34029)
0806 02:13:00.057889 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:00.057893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:00.058085 (+ 192us) server_negotiation.cc:407] Connection header received
0806 02:13:00.058133 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:00.058135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:00.058171 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:00.058322 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:13:00.058689 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.059187 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.060093 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.060308 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.060417 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:00.060665 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:13:00.060733 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:13:00.060820 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:00.060903 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 02:13:00.060947 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 02:13:00.071399 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:00.068100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:43961 (local address 127.18.70.66:34029)
0806 02:13:00.068226 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:00.068230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:00.068407 (+ 177us) server_negotiation.cc:407] Connection header received
0806 02:13:00.068561 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:00.068564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:00.068603 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:00.068677 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:13:00.068907 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.069365 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.070181 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.070524 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.070634 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:00.070934 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:13:00.071004 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:13:00.071078 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:00.071165 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 02:13:00.071229 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":38}
I20260806 02:13:00.071408 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:00.068043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:34029 (local address 127.18.70.66:43961)
0806 02:13:00.068330 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:00.068342 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:00.068453 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:00.068684 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:00.068687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:00.068693 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 02:13:00.068795 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:00.068805 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.069466 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.069471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:00.070067 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:00.070075 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.070632 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.070636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:00.070760 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:00.070800 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 02:13:00.071092 (+ 292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 02:13:00.071098 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 02:13:00.071156 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 02:13:00.071255 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":42,"thread_start_us":115,"threads_started":1}
I20260806 02:13:00.077391 26996 raft_consensus.cc:1261] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Refusing update from remote peer 9b84105c8d244dcba15fd88d1c771563: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:13:00.077541 26703 raft_consensus.cc:1261] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Refusing update from remote peer 9b84105c8d244dcba15fd88d1c771563: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:13:00.077832 27122 consensus_queue.cc:1035] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:13:00.078002 27126 consensus_queue.cc:1035] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:13:00.114229 26533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:13:00.114415 26533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:13:00.116530 26533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040: Sending DeleteTablet for 3 replicas of tablet 1bdf1c6bb5c24f2ebd9f03e369710404
I20260806 02:13:00.117134 26976 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bdf1c6bb5c24f2ebd9f03e369710404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:13:00 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:43504
I20260806 02:13:00.117168 26682 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bdf1c6bb5c24f2ebd9f03e369710404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:13:00 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:34906
I20260806 02:13:00.117336 27095 tablet_replica.cc:330] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4: stopping tablet replica
I20260806 02:13:00.117342 27096 tablet_replica.cc:330] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219: stopping tablet replica
I20260806 02:13:00.117449 27096 raft_consensus.cc:2229] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:13:00.117456 27095 raft_consensus.cc:2229] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:13:00.117558 27096 raft_consensus.cc:2258] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:13:00.117569 27095 raft_consensus.cc:2258] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:13:00.118278 27096 ts_tablet_manager.cc:1905] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:13:00.118288 27095 ts_tablet_manager.cc:1905] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:13:00.121215 27096 ts_tablet_manager.cc:1918] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:13:00.121306 27096 log.cc:1196] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/wal/wals/1bdf1c6bb5c24f2ebd9f03e369710404
I20260806 02:13:00.121567 27096 ts_tablet_manager.cc:1939] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 979fdbf695d64161b91154eb270df219: Deleting consensus metadata
I20260806 02:13:00.121840 27095 ts_tablet_manager.cc:1918] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:13:00.121917 27095 log.cc:1196] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/1bdf1c6bb5c24f2ebd9f03e369710404
I20260806 02:13:00.122184 27095 ts_tablet_manager.cc:1939] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 52495f3b1eff44f9b97d82b68f6b88f4: Deleting consensus metadata
I20260806 02:13:00.122228 26827 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bdf1c6bb5c24f2ebd9f03e369710404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:13:00 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:41966
I20260806 02:13:00.122439 27097 tablet_replica.cc:330] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563: stopping tablet replica
I20260806 02:13:00.122550 27097 raft_consensus.cc:2229] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:13:00.122851 27097 raft_consensus.cc:2258] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:13:00.122936 26521 catalog_manager.cc:4841] TS 52495f3b1eff44f9b97d82b68f6b88f4 (127.18.70.67:43521): tablet 1bdf1c6bb5c24f2ebd9f03e369710404 (table test-table [id=669fd5b43d1a4a4fbda07791f8a2575d]) successfully deleted
I20260806 02:13:00.123637 27097 ts_tablet_manager.cc:1905] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:13:00.126294 27097 ts_tablet_manager.cc:1918] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:13:00.126380 27097 log.cc:1196] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/1bdf1c6bb5c24f2ebd9f03e369710404
I20260806 02:13:00.126636 27097 ts_tablet_manager.cc:1939] T 1bdf1c6bb5c24f2ebd9f03e369710404 P 9b84105c8d244dcba15fd88d1c771563: Deleting consensus metadata
I20260806 02:13:00.126947 26519 catalog_manager.cc:4841] TS 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461): tablet 1bdf1c6bb5c24f2ebd9f03e369710404 (table test-table [id=669fd5b43d1a4a4fbda07791f8a2575d]) successfully deleted
I20260806 02:13:00.127748 26520 catalog_manager.cc:4841] TS 9b84105c8d244dcba15fd88d1c771563 (127.18.70.66:34029): tablet 1bdf1c6bb5c24f2ebd9f03e369710404 (table test-table [id=669fd5b43d1a4a4fbda07791f8a2575d]) successfully deleted
Aug 06 02:13:00 dist-test-slave-6k22 krb5kdc[26487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 02:13:00 dist-test-slave-6k22 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:00 dist-test-slave-6k22 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 02:13:00.150210 27112 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:00.143249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53426 (local address 127.18.70.126:44033)
0806 02:13:00.143394 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:00.143400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:00.143532 (+ 132us) server_negotiation.cc:407] Connection header received
0806 02:13:00.143742 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:00.143746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:00.143801 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:00.143895 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:00.144604 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.145487 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.146322 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.146696 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.146806 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:00.150091 (+ 3285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53426: BlockingRecv error: recv got EOF from 127.0.0.1:53426 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 02:13:00 dist-test-slave-6k22 krb5kdc[26487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM
I20260806 02:13:00.160679 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:00.151924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.18.70.126:44033)
0806 02:13:00.152042 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:00.152045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:00.152057 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:13:00.152087 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:00.152089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:00.152122 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:00.152186 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:00.152918 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.153393 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.154444 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.154656 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.154731 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:00.157369 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:00.157373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:00.157376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:13:00.157402 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:00.159021 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:00.159437 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:00.159442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:00.159444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:00.159538 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:00.159826 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:00.159833 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:00.159834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:00.160063 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:00.160195 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:00.160398 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 02:13:00.160522 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 02:13:00.163816 26533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:13:00.164327 26533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:13:00.169521 26682 tablet_service.cc:1467] Processing CreateTablet for tablet 46f776538c394dfbb20fa5c117392828 (DEFAULT_TABLE table=test-table [id=855cca7dc43b46c6bfcb1af21e04bddb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:13:00.169531 26827 tablet_service.cc:1467] Processing CreateTablet for tablet 46f776538c394dfbb20fa5c117392828 (DEFAULT_TABLE table=test-table [id=855cca7dc43b46c6bfcb1af21e04bddb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:13:00.169656 26976 tablet_service.cc:1467] Processing CreateTablet for tablet 46f776538c394dfbb20fa5c117392828 (DEFAULT_TABLE table=test-table [id=855cca7dc43b46c6bfcb1af21e04bddb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:13:00.169871 26682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46f776538c394dfbb20fa5c117392828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:13:00.169899 26827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46f776538c394dfbb20fa5c117392828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:13:00.169919 26976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46f776538c394dfbb20fa5c117392828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:13:00.172544 27074 tablet_bootstrap.cc:492] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4: Bootstrap starting.
I20260806 02:13:00.172719 27071 tablet_bootstrap.cc:492] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219: Bootstrap starting.
I20260806 02:13:00.173678 27074 tablet_bootstrap.cc:654] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4: Neither blocks nor log segments found. Creating new log.
I20260806 02:13:00.173707 27071 tablet_bootstrap.cc:654] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219: Neither blocks nor log segments found. Creating new log.
I20260806 02:13:00.175832 27074 tablet_bootstrap.cc:492] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4: No bootstrap required, opened a new log
I20260806 02:13:00.175872 27072 tablet_bootstrap.cc:492] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563: Bootstrap starting.
I20260806 02:13:00.175954 27074 ts_tablet_manager.cc:1397] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:13:00.176466 27074 raft_consensus.cc:348] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.176589 27074 raft_consensus.cc:374] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:13:00.176627 27074 raft_consensus.cc:729] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52495f3b1eff44f9b97d82b68f6b88f4, State: Initialized, Role: FOLLOWER
I20260806 02:13:00.176770 27074 consensus_queue.cc:260] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.177028 27074 ts_tablet_manager.cc:1428] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:13:00.177433 27072 tablet_bootstrap.cc:654] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563: Neither blocks nor log segments found. Creating new log.
I20260806 02:13:00.180766 27072 tablet_bootstrap.cc:492] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563: No bootstrap required, opened a new log
I20260806 02:13:00.180858 27072 ts_tablet_manager.cc:1397] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260806 02:13:00.181341 27072 raft_consensus.cc:348] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.181459 27072 raft_consensus.cc:374] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:13:00.181489 27072 raft_consensus.cc:729] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b84105c8d244dcba15fd88d1c771563, State: Initialized, Role: FOLLOWER
I20260806 02:13:00.181612 27072 consensus_queue.cc:260] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.181918 27072 ts_tablet_manager.cc:1428] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:13:00.182485 27071 tablet_bootstrap.cc:492] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219: No bootstrap required, opened a new log
I20260806 02:13:00.182610 27071 ts_tablet_manager.cc:1397] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260806 02:13:00.183027 27071 raft_consensus.cc:348] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.183151 27071 raft_consensus.cc:374] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:13:00.183200 27071 raft_consensus.cc:729] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979fdbf695d64161b91154eb270df219, State: Initialized, Role: FOLLOWER
I20260806 02:13:00.183339 27071 consensus_queue.cc:260] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.183670 27071 ts_tablet_manager.cc:1428] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:13:00.358021 27090 raft_consensus.cc:482] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:13:00.358209 27090 raft_consensus.cc:504] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.358670 27090 leader_election.cc:288] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461), 9b84105c8d244dcba15fd88d1c771563 (127.18.70.66:34029)
I20260806 02:13:00.359135 26849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46f776538c394dfbb20fa5c117392828" candidate_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b84105c8d244dcba15fd88d1c771563" is_pre_election: true
I20260806 02:13:00.359146 26702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46f776538c394dfbb20fa5c117392828" candidate_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979fdbf695d64161b91154eb270df219" is_pre_election: true
I20260806 02:13:00.359273 26849 raft_consensus.cc:2454] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52495f3b1eff44f9b97d82b68f6b88f4 in term 0.
I20260806 02:13:00.359315 26702 raft_consensus.cc:2454] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52495f3b1eff44f9b97d82b68f6b88f4 in term 0.
I20260806 02:13:00.359651 26923 leader_election.cc:302] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52495f3b1eff44f9b97d82b68f6b88f4, 9b84105c8d244dcba15fd88d1c771563; no voters:
I20260806 02:13:00.359889 27090 raft_consensus.cc:2781] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:13:00.359954 27090 raft_consensus.cc:482] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:13:00.359992 27090 raft_consensus.cc:3037] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:00.360997 27090 raft_consensus.cc:504] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.361318 27090 leader_election.cc:288] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [CANDIDATE]: Term 1 election: Requested vote from peers 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461), 9b84105c8d244dcba15fd88d1c771563 (127.18.70.66:34029)
I20260806 02:13:00.361634 26702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46f776538c394dfbb20fa5c117392828" candidate_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979fdbf695d64161b91154eb270df219"
I20260806 02:13:00.361656 26849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46f776538c394dfbb20fa5c117392828" candidate_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b84105c8d244dcba15fd88d1c771563"
I20260806 02:13:00.361733 26702 raft_consensus.cc:3037] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:00.361801 26849 raft_consensus.cc:3037] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:00.362757 26849 raft_consensus.cc:2454] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52495f3b1eff44f9b97d82b68f6b88f4 in term 1.
I20260806 02:13:00.363116 26923 leader_election.cc:302] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52495f3b1eff44f9b97d82b68f6b88f4, 9b84105c8d244dcba15fd88d1c771563; no voters:
I20260806 02:13:00.363144 26702 raft_consensus.cc:2454] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52495f3b1eff44f9b97d82b68f6b88f4 in term 1.
I20260806 02:13:00.363286 27090 raft_consensus.cc:2781] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:13:00.363471 27090 raft_consensus.cc:686] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 LEADER]: Becoming Leader. State: Replica: 52495f3b1eff44f9b97d82b68f6b88f4, State: Running, Role: LEADER
I20260806 02:13:00.363685 27090 consensus_queue.cc:237] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.365561 26532 catalog_manager.cc:5495] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52495f3b1eff44f9b97d82b68f6b88f4 (127.18.70.67). New cstate: current_term: 1 leader_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } health_report { overall_health: HEALTHY } } }
I20260806 02:13:00.418840 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:00.415834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.18.70.67:43521)
0806 02:13:00.415963 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:00.415968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:00.415987 (+ 19us) server_negotiation.cc:407] Connection header received
0806 02:13:00.416037 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:00.416039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:00.416081 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:00.416146 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:13:00.416387 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.416875 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.417838 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.418099 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.418204 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:00.418446 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:13:00.418531 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:13:00.418607 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:00.418680 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 02:13:00.418726 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 02:13:00.420867 26849 raft_consensus.cc:1261] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Refusing update from remote peer 52495f3b1eff44f9b97d82b68f6b88f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:13:00.420989 26702 raft_consensus.cc:1261] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Refusing update from remote peer 52495f3b1eff44f9b97d82b68f6b88f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:13:00.421345 27146 consensus_queue.cc:1035] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:13:00.421506 27090 consensus_queue.cc:1035] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:13:00.429011 26532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:13:00.429164 26532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:13:00.431008 26532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040: Sending DeleteTablet for 3 replicas of tablet 46f776538c394dfbb20fa5c117392828
I20260806 02:13:00.431658 26682 tablet_service.cc:1514] Processing DeleteTablet for tablet 46f776538c394dfbb20fa5c117392828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:13:00 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:34906
I20260806 02:13:00.431711 26827 tablet_service.cc:1514] Processing DeleteTablet for tablet 46f776538c394dfbb20fa5c117392828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:13:00 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:41966
I20260806 02:13:00.431859 26976 tablet_service.cc:1514] Processing DeleteTablet for tablet 46f776538c394dfbb20fa5c117392828 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:13:00 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:43504
I20260806 02:13:00.431910 27097 tablet_replica.cc:330] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563: stopping tablet replica
I20260806 02:13:00.431918 27096 tablet_replica.cc:330] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219: stopping tablet replica
I20260806 02:13:00.432000 27095 tablet_replica.cc:330] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4: stopping tablet replica
I20260806 02:13:00.432008 27096 raft_consensus.cc:2229] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:13:00.432029 27097 raft_consensus.cc:2229] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:13:00.432094 27096 raft_consensus.cc:2258] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:13:00.432099 27095 raft_consensus.cc:2229] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:13:00.432145 27097 raft_consensus.cc:2258] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:13:00.432286 27095 raft_consensus.cc:2258] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:13:00.432884 26532 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 02:13:00.433075 27096 ts_tablet_manager.cc:1905] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:13:00.433101 27097 ts_tablet_manager.cc:1905] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 02:13:00.433277 26532 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:13:00.435446 27096 ts_tablet_manager.cc:1918] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:13:00.435530 27096 log.cc:1196] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/wal/wals/46f776538c394dfbb20fa5c117392828
I20260806 02:13:00.435614 27097 ts_tablet_manager.cc:1918] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:13:00.435688 27097 log.cc:1196] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/46f776538c394dfbb20fa5c117392828
I20260806 02:13:00.435853 27096 ts_tablet_manager.cc:1939] T 46f776538c394dfbb20fa5c117392828 P 979fdbf695d64161b91154eb270df219: Deleting consensus metadata
I20260806 02:13:00.435979 27097 ts_tablet_manager.cc:1939] T 46f776538c394dfbb20fa5c117392828 P 9b84105c8d244dcba15fd88d1c771563: Deleting consensus metadata
I20260806 02:13:00.436484 26519 catalog_manager.cc:4841] TS 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461): tablet 46f776538c394dfbb20fa5c117392828 (table test-table [id=855cca7dc43b46c6bfcb1af21e04bddb]) successfully deleted
I20260806 02:13:00.437224 26520 catalog_manager.cc:4841] TS 9b84105c8d244dcba15fd88d1c771563 (127.18.70.66:34029): tablet 46f776538c394dfbb20fa5c117392828 (table test-table [id=855cca7dc43b46c6bfcb1af21e04bddb]) successfully deleted
I20260806 02:13:00.437312 27095 ts_tablet_manager.cc:1905] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:13:00.439358 27095 ts_tablet_manager.cc:1918] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:13:00.439420 27095 log.cc:1196] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/46f776538c394dfbb20fa5c117392828
I20260806 02:13:00.439674 27095 ts_tablet_manager.cc:1939] T 46f776538c394dfbb20fa5c117392828 P 52495f3b1eff44f9b97d82b68f6b88f4: Deleting consensus metadata
I20260806 02:13:00.440481 26521 catalog_manager.cc:4841] TS 52495f3b1eff44f9b97d82b68f6b88f4 (127.18.70.67:43521): tablet 46f776538c394dfbb20fa5c117392828 (table test-table [id=855cca7dc43b46c6bfcb1af21e04bddb]) successfully deleted
I20260806 02:13:00.440759 26827 tablet_service.cc:1467] Processing CreateTablet for tablet ea5c6151cfed42adb5760e33e23b82a9 (DEFAULT_TABLE table=test-table [id=38a3204ce4154758b611fa50731e13a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:13:00.440757 26976 tablet_service.cc:1467] Processing CreateTablet for tablet ea5c6151cfed42adb5760e33e23b82a9 (DEFAULT_TABLE table=test-table [id=38a3204ce4154758b611fa50731e13a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:13:00.440769 26682 tablet_service.cc:1467] Processing CreateTablet for tablet ea5c6151cfed42adb5760e33e23b82a9 (DEFAULT_TABLE table=test-table [id=38a3204ce4154758b611fa50731e13a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:13:00.440986 26827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea5c6151cfed42adb5760e33e23b82a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:13:00.440987 26682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea5c6151cfed42adb5760e33e23b82a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:13:00.440987 26976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea5c6151cfed42adb5760e33e23b82a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:13:00.442891 27072 tablet_bootstrap.cc:492] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563: Bootstrap starting.
I20260806 02:13:00.443033 27071 tablet_bootstrap.cc:492] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219: Bootstrap starting.
I20260806 02:13:00.443783 27072 tablet_bootstrap.cc:654] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563: Neither blocks nor log segments found. Creating new log.
I20260806 02:13:00.443925 27071 tablet_bootstrap.cc:654] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219: Neither blocks nor log segments found. Creating new log.
I20260806 02:13:00.444543 27074 tablet_bootstrap.cc:492] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4: Bootstrap starting.
I20260806 02:13:00.444947 27072 tablet_bootstrap.cc:492] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563: No bootstrap required, opened a new log
I20260806 02:13:00.445050 27072 ts_tablet_manager.cc:1397] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 02:13:00.445127 27071 tablet_bootstrap.cc:492] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219: No bootstrap required, opened a new log
I20260806 02:13:00.445238 27071 ts_tablet_manager.cc:1397] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 02:13:00.445399 27074 tablet_bootstrap.cc:654] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4: Neither blocks nor log segments found. Creating new log.
I20260806 02:13:00.445513 27072 raft_consensus.cc:348] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.445636 27072 raft_consensus.cc:374] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:13:00.445677 27072 raft_consensus.cc:729] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b84105c8d244dcba15fd88d1c771563, State: Initialized, Role: FOLLOWER
I20260806 02:13:00.445696 27071 raft_consensus.cc:348] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.445832 27071 raft_consensus.cc:374] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:13:00.445823 27072 consensus_queue.cc:260] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.445878 27071 raft_consensus.cc:729] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979fdbf695d64161b91154eb270df219, State: Initialized, Role: FOLLOWER
I20260806 02:13:00.446003 27071 consensus_queue.cc:260] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.446086 27072 ts_tablet_manager.cc:1428] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:13:00.446282 27071 ts_tablet_manager.cc:1428] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:13:00.446898 27074 tablet_bootstrap.cc:492] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4: No bootstrap required, opened a new log
I20260806 02:13:00.446990 27074 ts_tablet_manager.cc:1397] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 02:13:00.447386 27074 raft_consensus.cc:348] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.447500 27074 raft_consensus.cc:374] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:13:00.447530 27074 raft_consensus.cc:729] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52495f3b1eff44f9b97d82b68f6b88f4, State: Initialized, Role: FOLLOWER
I20260806 02:13:00.447670 27074 consensus_queue.cc:260] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.448010 27074 ts_tablet_manager.cc:1428] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 02:13:00.520015 27151 raft_consensus.cc:482] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:13:00.520270 27151 raft_consensus.cc:504] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.520859 27151 leader_election.cc:288] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461), 9b84105c8d244dcba15fd88d1c771563 (127.18.70.66:34029)
I20260806 02:13:00.521219 26702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea5c6151cfed42adb5760e33e23b82a9" candidate_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979fdbf695d64161b91154eb270df219" is_pre_election: true
I20260806 02:13:00.521307 26849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea5c6151cfed42adb5760e33e23b82a9" candidate_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b84105c8d244dcba15fd88d1c771563" is_pre_election: true
I20260806 02:13:00.521396 26702 raft_consensus.cc:2454] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52495f3b1eff44f9b97d82b68f6b88f4 in term 0.
I20260806 02:13:00.521483 26849 raft_consensus.cc:2454] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52495f3b1eff44f9b97d82b68f6b88f4 in term 0.
I20260806 02:13:00.521695 26922 leader_election.cc:302] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52495f3b1eff44f9b97d82b68f6b88f4, 979fdbf695d64161b91154eb270df219; no voters:
I20260806 02:13:00.521873 27151 raft_consensus.cc:2781] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:13:00.521936 27151 raft_consensus.cc:482] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:13:00.521989 27151 raft_consensus.cc:3037] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:00.523023 27151 raft_consensus.cc:504] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.523383 27151 leader_election.cc:288] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [CANDIDATE]: Term 1 election: Requested vote from peers 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461), 9b84105c8d244dcba15fd88d1c771563 (127.18.70.66:34029)
I20260806 02:13:00.523684 26702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea5c6151cfed42adb5760e33e23b82a9" candidate_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979fdbf695d64161b91154eb270df219"
I20260806 02:13:00.523813 26702 raft_consensus.cc:3037] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:00.523800 26849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea5c6151cfed42adb5760e33e23b82a9" candidate_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b84105c8d244dcba15fd88d1c771563"
I20260806 02:13:00.523927 26849 raft_consensus.cc:3037] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:00.525025 26702 raft_consensus.cc:2454] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52495f3b1eff44f9b97d82b68f6b88f4 in term 1.
I20260806 02:13:00.525133 26849 raft_consensus.cc:2454] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52495f3b1eff44f9b97d82b68f6b88f4 in term 1.
I20260806 02:13:00.525344 26922 leader_election.cc:302] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52495f3b1eff44f9b97d82b68f6b88f4, 979fdbf695d64161b91154eb270df219; no voters:
I20260806 02:13:00.525548 27151 raft_consensus.cc:2781] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:13:00.525640 27151 raft_consensus.cc:686] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 LEADER]: Becoming Leader. State: Replica: 52495f3b1eff44f9b97d82b68f6b88f4, State: Running, Role: LEADER
I20260806 02:13:00.525779 27151 consensus_queue.cc:237] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } }
I20260806 02:13:00.527359 26534 catalog_manager.cc:5495] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52495f3b1eff44f9b97d82b68f6b88f4 (127.18.70.67). New cstate: current_term: 1 leader_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52495f3b1eff44f9b97d82b68f6b88f4" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 43521 } health_report { overall_health: HEALTHY } } }
I20260806 02:13:00.556986 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:00.553984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:54141 (local address 127.18.70.66:34029)
0806 02:13:00.554138 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:00.554142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:00.554341 (+ 199us) server_negotiation.cc:407] Connection header received
0806 02:13:00.554427 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:00.554429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:00.554472 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:00.554533 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:13:00.554729 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.555243 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.556120 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.556335 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.556416 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:00.556654 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:13:00.556715 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:13:00.556767 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:00.556846 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 02:13:00.556884 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 02:13:00.557008 27158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:00.553927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:34029 (local address 127.18.70.67:54141)
0806 02:13:00.554258 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:00.554272 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:00.554360 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:00.554555 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:00.554557 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:00.554563 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 02:13:00.554659 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:00.554663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.555338 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.555340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:00.556016 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:00.556026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.556435 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.556437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:00.556498 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:00.556543 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 02:13:00.556787 (+ 244us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 02:13:00.556792 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 02:13:00.556840 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 02:13:00.556901 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":140,"threads_started":1}
I20260806 02:13:00.562378 26849 raft_consensus.cc:1261] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Refusing update from remote peer 52495f3b1eff44f9b97d82b68f6b88f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:13:00.562505 26702 raft_consensus.cc:1261] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Refusing update from remote peer 52495f3b1eff44f9b97d82b68f6b88f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:13:00.562723 27090 consensus_queue.cc:1035] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b84105c8d244dcba15fd88d1c771563" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 34029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:13:00.562891 27151 consensus_queue.cc:1035] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "979fdbf695d64161b91154eb270df219" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 39461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:13:00.578765 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:00.575531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:43521 (local address 127.18.70.66:53149)
0806 02:13:00.575759 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:00.575774 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:00.575895 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:00.576125 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:00.576128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:00.576136 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 02:13:00.576234 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:00.576240 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.576980 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.576985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:00.577587 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:00.577596 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.578068 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.578071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:00.578158 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:00.578191 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 02:13:00.578499 (+ 308us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 02:13:00.578506 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 02:13:00.578564 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 02:13:00.578643 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
I20260806 02:13:00.578753 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:00.575583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:53149 (local address 127.18.70.67:43521)
0806 02:13:00.575693 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:00.575699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:00.575856 (+ 157us) server_negotiation.cc:407] Connection header received
0806 02:13:00.575974 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:00.575977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:00.576020 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:00.576102 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:13:00.576352 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.576869 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.577711 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:00.577954 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:00.578066 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:00.578315 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:13:00.578399 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:13:00.578483 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:00.578575 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 02:13:00.578624 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 02:13:00.597848 26532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:13:00.597980 26532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:13:00.599958 26532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce1b120a62474933b5ec52ffb8b7e040: Sending DeleteTablet for 3 replicas of tablet ea5c6151cfed42adb5760e33e23b82a9
I20260806 02:13:00.600486 26682 tablet_service.cc:1514] Processing DeleteTablet for tablet ea5c6151cfed42adb5760e33e23b82a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:13:00 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:34906
I20260806 02:13:00.600486 26827 tablet_service.cc:1514] Processing DeleteTablet for tablet ea5c6151cfed42adb5760e33e23b82a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:13:00 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:41966
I20260806 02:13:00.600653 27097 tablet_replica.cc:330] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563: stopping tablet replica
I20260806 02:13:00.600682 26976 tablet_service.cc:1514] Processing DeleteTablet for tablet ea5c6151cfed42adb5760e33e23b82a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:13:00 UTC) from {username='oryx', principal='oryx/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:43504
I20260806 02:13:00.600716 27096 tablet_replica.cc:330] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219: stopping tablet replica
I20260806 02:13:00.600788 27097 raft_consensus.cc:2229] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:13:00.600823 27095 tablet_replica.cc:330] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4: stopping tablet replica
I20260806 02:13:00.600858 27096 raft_consensus.cc:2229] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:13:00.600919 27097 raft_consensus.cc:2258] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:13:00.600947 27095 raft_consensus.cc:2229] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:13:00.600987 27096 raft_consensus.cc:2258] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:13:00.601121 27095 raft_consensus.cc:2258] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:13:00.601624 27096 ts_tablet_manager.cc:1905] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:13:00.601648 27097 ts_tablet_manager.cc:1905] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:13:00.603698 27097 ts_tablet_manager.cc:1918] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:13:00.603698 27096 ts_tablet_manager.cc:1918] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:13:00.603781 27096 log.cc:1196] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-0/wal/wals/ea5c6151cfed42adb5760e33e23b82a9
I20260806 02:13:00.603788 27097 log.cc:1196] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/ea5c6151cfed42adb5760e33e23b82a9
I20260806 02:13:00.604045 27096 ts_tablet_manager.cc:1939] T ea5c6151cfed42adb5760e33e23b82a9 P 979fdbf695d64161b91154eb270df219: Deleting consensus metadata
I20260806 02:13:00.604045 27097 ts_tablet_manager.cc:1939] T ea5c6151cfed42adb5760e33e23b82a9 P 9b84105c8d244dcba15fd88d1c771563: Deleting consensus metadata
I20260806 02:13:00.604192 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 26615
I20260806 02:13:00.604636 27095 ts_tablet_manager.cc:1905] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:13:00.607250 27095 ts_tablet_manager.cc:1918] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 02:13:00.607316 27095 log.cc:1196] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/ea5c6151cfed42adb5760e33e23b82a9
I20260806 02:13:00.607496 27095 ts_tablet_manager.cc:1939] T ea5c6151cfed42adb5760e33e23b82a9 P 52495f3b1eff44f9b97d82b68f6b88f4: Deleting consensus metadata
I20260806 02:13:00.613947 26520 catalog_manager.cc:4841] TS 9b84105c8d244dcba15fd88d1c771563 (127.18.70.66:34029): tablet ea5c6151cfed42adb5760e33e23b82a9 (table test-table [id=38a3204ce4154758b611fa50731e13a6]) successfully deleted
I20260806 02:13:00.614529 26521 catalog_manager.cc:4841] TS 52495f3b1eff44f9b97d82b68f6b88f4 (127.18.70.67:43521): tablet ea5c6151cfed42adb5760e33e23b82a9 (table test-table [id=38a3204ce4154758b611fa50731e13a6]) successfully deleted
W20260806 02:13:00.618314 26519 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:39461 (error 108)
I20260806 02:13:00.619185 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 26760
I20260806 02:13:00.619526 27166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:00.619065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:39461 (local address 127.0.0.1:34918)
0806 02:13:00.619314 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:00.619396 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:39461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":50,"threads_started":1}
W20260806 02:13:00.620155 26519 catalog_manager.cc:4584] TS 979fdbf695d64161b91154eb270df219 (127.18.70.65:39461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea5c6151cfed42adb5760e33e23b82a9: Network error: Client connection negotiation failed: client connection to 127.18.70.65:39461: connect: Connection refused (error 111)
I20260806 02:13:00.629256 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 26905
I20260806 02:13:00.637154 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 26500
2026-08-06T02:13:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3572 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:13:00 dist-test-slave-6k22 krb5kdc[27169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:13:00 dist-test-slave-6k22 krb5kdc[27169](info): set up 2 sockets
Aug 06 02:13:00 dist-test-slave-6k22 krb5kdc[27169](info): commencing operation
krb5kdc: starting...
W20260806 02:13:01.698463 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:13:01 dist-test-slave-6k22 krb5kdc[27169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:13:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:13:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:13:01.824512 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:34781
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:39679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:34781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:13:01.936580 27182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:01.936800 27182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:01.936853 27182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:01.941143 27182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:13:01.941195 27182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:01.941212 27182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:01.941227 27182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:13:01.941278 27182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:13:01.946130 27182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:39679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:34781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:34781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:01.947117 27182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:01.948128 27182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:01.956391 27188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:01.956413 27187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:01.956600 27182 server_base.cc:1034] running on GCE node
W20260806 02:13:01.956794 27190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:01.957312 27182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:01.958372 27182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:01.959638 27182 hybrid_clock.cc:648] HybridClock initialized: now 1785982381959599 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T02:13:01Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1332 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:13:02 dist-test-slave-6k22 krb5kdc[27195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:13:02 dist-test-slave-6k22 krb5kdc[27195](info): set up 2 sockets
Aug 06 02:13:02 dist-test-slave-6k22 krb5kdc[27195](info): commencing operation
krb5kdc: starting...
W20260806 02:13:03.033298 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:13:03 dist-test-slave-6k22 krb5kdc[27195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:13:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:13:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:13:03.154891 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:44741
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:37955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:44741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:13:03.263033 27208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:03.263316 27208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:03.263394 27208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:03.267319 27208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:13:03.267424 27208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:03.267463 27208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:03.267496 27208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:13:03.267524 27208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:13:03.272644 27208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:44741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:44741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:03.273690 27208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:03.274689 27208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:03.283177 27213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:03.283218 27214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:03.283214 27216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:03.283438 27208 server_base.cc:1034] running on GCE node
I20260806 02:13:03.284075 27208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:03.285146 27208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:03.286345 27208 hybrid_clock.cc:648] HybridClock initialized: now 1785982383286330 us; error 35 us; skew 500 ppm
Aug 06 02:13:03 dist-test-slave-6k22 krb5kdc[27195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982383, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:13:03.289067 27208 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:13:03.289983 27208 webserver.cc:460] Webserver started at http://127.18.70.126:37867/ using document root <none> and password file <none>
I20260806 02:13:03.290549 27208 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:03.290642 27208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:03.290875 27208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:03.292963 27208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "8696c0bcd6d4490a80c448af6e3b0d7c"
format_stamp: "Formatted at 2026-08-06 02:13:03 on dist-test-slave-6k22"
server_key: "e5a93b547d0dfcc8483e2a7e030f3604"
server_key_iv: "ee1ec1b1050eca754a590ce03a9d3b96"
server_key_version: "encryptionkey@0"
I20260806 02:13:03.293569 27208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "8696c0bcd6d4490a80c448af6e3b0d7c"
format_stamp: "Formatted at 2026-08-06 02:13:03 on dist-test-slave-6k22"
server_key: "e5a93b547d0dfcc8483e2a7e030f3604"
server_key_iv: "ee1ec1b1050eca754a590ce03a9d3b96"
server_key_version: "encryptionkey@0"
I20260806 02:13:03.297238 27208 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:13:03.299547 27223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:03.300794 27208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:13:03.300941 27208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "8696c0bcd6d4490a80c448af6e3b0d7c"
format_stamp: "Formatted at 2026-08-06 02:13:03 on dist-test-slave-6k22"
server_key: "e5a93b547d0dfcc8483e2a7e030f3604"
server_key_iv: "ee1ec1b1050eca754a590ce03a9d3b96"
server_key_version: "encryptionkey@0"
I20260806 02:13:03.301054 27208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:03.321012 27208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:03.323931 27208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:03.324136 27208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:03.331336 27208 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:44741
I20260806 02:13:03.331346 27279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:44741 every 8 connection(s)
I20260806 02:13:03.332394 27208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:13:03.335107 27280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:13:03.340695 27280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c: Bootstrap starting.
I20260806 02:13:03.341153 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 27208
I20260806 02:13:03.341295 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:13:03.341586 18713 external_mini_cluster.cc:1428] Setting key cf83117e5727d6e26214005429251c2e
I20260806 02:13:03.343492 27280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c: Neither blocks nor log segments found. Creating new log.
I20260806 02:13:03.344313 27280 log.cc:824] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c: Log is configured to *not* fsync() on all Append() calls
I20260806 02:13:03.346154 27280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c: No bootstrap required, opened a new log
I20260806 02:13:03.349535 27280 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44741 } }
I20260806 02:13:03.349798 27280 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:13:03.349882 27280 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8696c0bcd6d4490a80c448af6e3b0d7c, State: Initialized, Role: FOLLOWER
I20260806 02:13:03.350416 27280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44741 } }
I20260806 02:13:03.350589 27280 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:13:03.350654 27280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 06 02:13:03 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:13:03.350771 27280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:03.351925 27280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44741 } }
I20260806 02:13:03.352281 27280 leader_election.cc:302] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8696c0bcd6d4490a80c448af6e3b0d7c; no voters:
I20260806 02:13:03.352586 27280 leader_election.cc:288] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:13:03.352725 27285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:13:03.352959 27285 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 1 LEADER]: Becoming Leader. State: Replica: 8696c0bcd6d4490a80c448af6e3b0d7c, State: Running, Role: LEADER
I20260806 02:13:03.353325 27285 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44741 } }
I20260806 02:13:03.354076 27280 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:13:03.354602 27285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8696c0bcd6d4490a80c448af6e3b0d7c. Latest consensus state: current_term: 1 leader_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44741 } } }
I20260806 02:13:03.354744 27285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [sys.catalog]: This master's current role is: LEADER
I20260806 02:13:03.355067 27287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44741 } } }
I20260806 02:13:03.355177 27287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [sys.catalog]: This master's current role is: LEADER
I20260806 02:13:03.357277 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:03.342933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48834 (local address 127.18.70.126:44741)
0806 02:13:03.343547 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:03.343589 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:03.343637 (+ 48us) server_negotiation.cc:407] Connection header received
0806 02:13:03.344342 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:03.344359 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:03.344728 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:03.345064 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:03.345893 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:03.346906 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:03.347842 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:03.348348 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:03.348529 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:03.351332 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:03.351338 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:03.351351 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:13:03.351377 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:03.353633 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:03.354197 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:03.354203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:03.354209 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:03.354359 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:03.354742 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:03.354745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:03.354746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:03.355351 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:03.355591 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:03.355918 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 02:13:03.356252 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":119,"threads_started":1}
W20260806 02:13:03.359277 27300 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:13:03.359369 27300 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:13:03.359438 27289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:13:03.360325 27289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:13:03.366192 27289 catalog_manager.cc:1347] Generated new cluster ID: 446f0c2d2cea49498a69e9760581dff2
I20260806 02:13:03.366266 27289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:13:03.376457 27289 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:13:03.377810 27289 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:13:03.386516 27289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c: Generated new TSK 0
I20260806 02:13:03.387403 27289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:13:03.418995 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:44741
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:37955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:13:03.532202 27306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:03.532495 27306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:03.532572 27306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:03.536480 27306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:03.536568 27306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:03.536659 27306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:13:03.542215 27306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:44741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:03.543329 27306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:03.544423 27306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:03.554129 27313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:03.554189 27311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:03.554189 27315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:03.554383 27306 server_base.cc:1034] running on GCE node
I20260806 02:13:03.554849 27306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:03.555675 27306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:03.556872 27306 hybrid_clock.cc:648] HybridClock initialized: now 1785982383556860 us; error 28 us; skew 500 ppm
Aug 06 02:13:03 dist-test-slave-6k22 krb5kdc[27195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982383, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:13:03.559846 27306 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:13:03.560717 27306 webserver.cc:460] Webserver started at http://127.18.70.65:40691/ using document root <none> and password file <none>
I20260806 02:13:03.561291 27306 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:03.561394 27306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:03.561661 27306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:03.563789 27306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "49cc7e3141cd4c7fa20e1b459996af81"
format_stamp: "Formatted at 2026-08-06 02:13:03 on dist-test-slave-6k22"
server_key: "09c59033acedea795d65eb4533e57bd1"
server_key_iv: "b58b5073ed9e681a525183cff6504aef"
server_key_version: "encryptionkey@0"
I20260806 02:13:03.564388 27306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "49cc7e3141cd4c7fa20e1b459996af81"
format_stamp: "Formatted at 2026-08-06 02:13:03 on dist-test-slave-6k22"
server_key: "09c59033acedea795d65eb4533e57bd1"
server_key_iv: "b58b5073ed9e681a525183cff6504aef"
server_key_version: "encryptionkey@0"
I20260806 02:13:03.568462 27306 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 02:13:03.570825 27322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:03.572074 27306 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:13:03.572207 27306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "49cc7e3141cd4c7fa20e1b459996af81"
format_stamp: "Formatted at 2026-08-06 02:13:03 on dist-test-slave-6k22"
server_key: "09c59033acedea795d65eb4533e57bd1"
server_key_iv: "b58b5073ed9e681a525183cff6504aef"
server_key_version: "encryptionkey@0"
I20260806 02:13:03.572319 27306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:03.603612 27306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:03.607102 27306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:03.607297 27306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:03.607975 27306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:03.608844 27306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:03.608891 27306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:03.608960 27306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:03.608991 27306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:03.617725 27306 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:43333
I20260806 02:13:03.617745 27439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:43333 every 8 connection(s)
I20260806 02:13:03.618654 27306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:13:03.626461 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 27306
I20260806 02:13:03.626622 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:13:03.626976 18713 external_mini_cluster.cc:1428] Setting key 23efba1986c7c053774fc16f19cf51fb
Aug 06 02:13:03 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982383, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:13:03.632361 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:03.620563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:44997 (local address 127.18.70.126:44741)
0806 02:13:03.620692 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:03.620696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:03.621729 (+ 1033us) server_negotiation.cc:407] Connection header received
0806 02:13:03.622569 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:03.622571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:03.622616 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:03.622688 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:03.623876 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:03.624398 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:03.625542 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:03.625767 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:03.625847 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:03.628581 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:03.628585 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:03.628588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:13:03.628615 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:03.630298 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:03.630822 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:03.630827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:03.630829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:03.630913 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:03.631269 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:03.631272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:03.631274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:03.631447 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:03.631557 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:03.632060 (+ 503us) server_negotiation.cc:299] Negotiation successful
0806 02:13:03.632181 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 02:13:03.633122 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:03.620929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:44997)
0806 02:13:03.621578 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:03.621619 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:03.622368 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:03.622848 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:03.622855 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:03.623181 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:03.623739 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:03.623754 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:03.624502 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:03.624505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:03.625429 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:03.625438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:03.625896 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:03.625899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:03.626034 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:03.626612 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:03.626638 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:03.628392 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:13:03.630419 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:13:03.630424 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:13:03.630434 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:13:03.630688 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:13:03.631010 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:13:03.631014 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:13:03.631016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:13:03.631157 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:13:03.631583 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:13:03.631592 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:13:03.631827 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 02:13:03.632016 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 02:13:03.632326 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":96,"threads_started":1}
I20260806 02:13:03.634490 27440 heartbeater.cc:347] Connected to a master server at 127.18.70.126:44741
I20260806 02:13:03.634721 27440 heartbeater.cc:464] Registering TS with master...
I20260806 02:13:03.635210 27440 heartbeater.cc:511] Master 127.18.70.126:44741 requested a full tablet report, sending...
I20260806 02:13:03.636782 27242 ts_manager.cc:194] Registered new tserver with Master: 49cc7e3141cd4c7fa20e1b459996af81 (127.18.70.65:43333)
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
I20260806 02:13:03.638021 27242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:44997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:13:03.656244 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:44741
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:37955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:13:03.768431 27445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:03.768684 27445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:03.768756 27445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:03.772523 27445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:03.772606 27445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:03.772709 27445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:13:03.777966 27445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:44741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:03.778962 27445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:03.780048 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:03.789374 27454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:03.789387 27452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:03.789389 27451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:03.789618 27445 server_base.cc:1034] running on GCE node
I20260806 02:13:03.789980 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:03.790627 27445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:03.791810 27445 hybrid_clock.cc:648] HybridClock initialized: now 1785982383791796 us; error 28 us; skew 500 ppm
Aug 06 02:13:03 dist-test-slave-6k22 krb5kdc[27195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982383, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:13:03.794446 27445 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:13:03.795347 27445 webserver.cc:460] Webserver started at http://127.18.70.66:42189/ using document root <none> and password file <none>
I20260806 02:13:03.795918 27445 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:03.795979 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:03.796139 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:03.798056 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "1a018cbf1f5a41b988f59c783d4f2303"
format_stamp: "Formatted at 2026-08-06 02:13:03 on dist-test-slave-6k22"
server_key: "9685bfd69c693ebe6d8bfb29c83e8484"
server_key_iv: "277db0d21c1829f005a48babd04ea3a9"
server_key_version: "encryptionkey@0"
I20260806 02:13:03.798563 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "1a018cbf1f5a41b988f59c783d4f2303"
format_stamp: "Formatted at 2026-08-06 02:13:03 on dist-test-slave-6k22"
server_key: "9685bfd69c693ebe6d8bfb29c83e8484"
server_key_iv: "277db0d21c1829f005a48babd04ea3a9"
server_key_version: "encryptionkey@0"
I20260806 02:13:03.802192 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 02:13:03.804574 27461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:03.805833 27445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:13:03.805932 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "1a018cbf1f5a41b988f59c783d4f2303"
format_stamp: "Formatted at 2026-08-06 02:13:03 on dist-test-slave-6k22"
server_key: "9685bfd69c693ebe6d8bfb29c83e8484"
server_key_iv: "277db0d21c1829f005a48babd04ea3a9"
server_key_version: "encryptionkey@0"
I20260806 02:13:03.806010 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:03.819094 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:03.821744 27445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:03.821888 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:03.822420 27445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:03.823202 27445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:03.823324 27445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:03.823419 27445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:03.823447 27445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:03.833748 27445 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:33133
I20260806 02:13:03.833762 27578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:33133 every 8 connection(s)
I20260806 02:13:03.834759 27445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:13:03.842772 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 27445
I20260806 02:13:03.842937 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:13:03.843252 18713 external_mini_cluster.cc:1428] Setting key bcaf95fcb643149447a1d103e214aeae
Aug 06 02:13:03 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982383, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:13:03.848646 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:03.836758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:57469 (local address 127.18.70.126:44741)
0806 02:13:03.836955 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:03.836961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:03.838021 (+ 1060us) server_negotiation.cc:407] Connection header received
0806 02:13:03.838875 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:03.838877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:03.838928 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:03.839016 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:03.840167 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:03.840703 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:03.841770 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:03.842001 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:03.842093 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:03.844917 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:03.844921 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:03.844923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:13:03.844944 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:03.846422 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:03.847101 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:03.847107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:03.847111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:03.847202 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:03.847546 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:03.847551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:03.847569 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:03.847802 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:03.847901 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:03.848394 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 02:13:03.848533 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 02:13:03.849382 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:03.837196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:57469)
0806 02:13:03.837870 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:03.837913 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:03.838674 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:03.839175 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:03.839181 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:03.839537 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:03.840026 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:03.840039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:03.840819 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:03.840821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:03.841654 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:03.841663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:03.842136 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:03.842139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:03.842282 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:03.842894 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:03.842937 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:03.844723 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:13:03.846552 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:13:03.846562 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:13:03.846579 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:13:03.846983 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:13:03.847304 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:13:03.847306 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:13:03.847308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:13:03.847447 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:13:03.847919 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:13:03.847925 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:13:03.848152 (+ 227us) client_negotiation.cc:769] Sending connection context
0806 02:13:03.848314 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 02:13:03.848581 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":115,"threads_started":1}
I20260806 02:13:03.850687 27579 heartbeater.cc:347] Connected to a master server at 127.18.70.126:44741
I20260806 02:13:03.850936 27579 heartbeater.cc:464] Registering TS with master...
I20260806 02:13:03.851426 27579 heartbeater.cc:511] Master 127.18.70.126:44741 requested a full tablet report, sending...
I20260806 02:13:03.852509 27242 ts_manager.cc:194] Registered new tserver with Master: 1a018cbf1f5a41b988f59c783d4f2303 (127.18.70.66:33133)
I20260806 02:13:03.853063 27242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:57469
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:13:03.874401 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:44741
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:37955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:13:03.987501 27584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:03.987814 27584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:03.987896 27584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:03.992250 27584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:03.992326 27584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:03.992421 27584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:13:03.998180 27584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:44741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:03.999185 27584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:04.000269 27584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:04.010020 27592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:04.010104 27589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:04.010233 27590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:04.010290 27584 server_base.cc:1034] running on GCE node
I20260806 02:13:04.010723 27584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:04.011399 27584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:04.012593 27584 hybrid_clock.cc:648] HybridClock initialized: now 1785982384012575 us; error 31 us; skew 500 ppm
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982384, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:13:04.015301 27584 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:13:04.016283 27584 webserver.cc:460] Webserver started at http://127.18.70.67:41145/ using document root <none> and password file <none>
I20260806 02:13:04.016855 27584 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:04.016947 27584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:04.017158 27584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:04.019164 27584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "63412137813148ce93fa162de656b354"
format_stamp: "Formatted at 2026-08-06 02:13:04 on dist-test-slave-6k22"
server_key: "9a4f70fd7e66d50f450b5911c15cd6a2"
server_key_iv: "8242ddb78b80eec02358258f4445500f"
server_key_version: "encryptionkey@0"
I20260806 02:13:04.019794 27584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "63412137813148ce93fa162de656b354"
format_stamp: "Formatted at 2026-08-06 02:13:04 on dist-test-slave-6k22"
server_key: "9a4f70fd7e66d50f450b5911c15cd6a2"
server_key_iv: "8242ddb78b80eec02358258f4445500f"
server_key_version: "encryptionkey@0"
I20260806 02:13:04.023531 27584 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 02:13:04.025885 27600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:04.027063 27584 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 02:13:04.027201 27584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "63412137813148ce93fa162de656b354"
format_stamp: "Formatted at 2026-08-06 02:13:04 on dist-test-slave-6k22"
server_key: "9a4f70fd7e66d50f450b5911c15cd6a2"
server_key_iv: "8242ddb78b80eec02358258f4445500f"
server_key_version: "encryptionkey@0"
I20260806 02:13:04.027326 27584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:04.042438 27584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:04.045131 27584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:04.045344 27584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:04.045991 27584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:04.046825 27584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:04.046895 27584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:04.046958 27584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:04.047008 27584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:04.056692 27584 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:45811
I20260806 02:13:04.056730 27715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:45811 every 8 connection(s)
I20260806 02:13:04.057605 27584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:13:04.060743 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 27584
I20260806 02:13:04.060889 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:13:04.061195 18713 external_mini_cluster.cc:1428] Setting key b0655ad7544cff256f21733beb76fc88
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982384, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:13:04.071278 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:04.059463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:39789 (local address 127.18.70.126:44741)
0806 02:13:04.059716 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:04.059719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:04.060776 (+ 1057us) server_negotiation.cc:407] Connection header received
0806 02:13:04.061944 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:04.061948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:04.062001 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:04.062092 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:04.063584 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.064108 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.065221 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.065433 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.065519 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.067979 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:04.067984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:04.067991 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:13:04.068015 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:04.069237 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:04.069749 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:04.069752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:04.069754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:04.069835 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:04.070183 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:04.070186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:04.070188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:04.070375 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:04.070492 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:04.070971 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 02:13:04.071099 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260806 02:13:04.072126 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:04.059892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:39789)
0806 02:13:04.060584 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:04.060645 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:04.061663 (+ 1018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:04.062295 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:04.062304 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:04.062822 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:04.063419 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.063440 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.064223 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.064225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.065102 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.065111 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.065570 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.065573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.065724 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.066282 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:04.066299 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:04.067814 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:13:04.069371 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:13:04.069377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:13:04.069391 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:13:04.069642 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:13:04.069928 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:13:04.069932 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:13:04.069934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:13:04.070078 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:13:04.070490 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:13:04.070497 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:13:04.070732 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 02:13:04.070930 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 02:13:04.071234 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":100,"threads_started":1}
I20260806 02:13:04.073227 27716 heartbeater.cc:347] Connected to a master server at 127.18.70.126:44741
I20260806 02:13:04.073443 27716 heartbeater.cc:464] Registering TS with master...
I20260806 02:13:04.074033 27716 heartbeater.cc:511] Master 127.18.70.126:44741 requested a full tablet report, sending...
I20260806 02:13:04.075150 27242 ts_manager.cc:194] Registered new tserver with Master: 63412137813148ce93fa162de656b354 (127.18.70.67:45811)
I20260806 02:13:04.075830 27242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:39789
I20260806 02:13:04.086743 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab.
Entry for principal oryx/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab.
WARNING: no policy specified for oryx/127.18.70.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.70.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab.
Entry for principal oryx/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab.
WARNING: no policy specified for oryx/127.18.70.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.70.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab.
Entry for principal oryx/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab.
I20260806 02:13:04.160436 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 27306
I20260806 02:13:04.169790 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 27445
I20260806 02:13:04.177654 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 27584
I20260806 02:13:04.184540 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 27208
I20260806 02:13:04.192095 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:44741
--webserver_interface=127.18.70.126
--webserver_port=37867
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:37955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:44741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:13:04.301502 27729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:04.301785 27729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:04.301862 27729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:04.305816 27729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:13:04.305894 27729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:04.305933 27729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:04.305991 27729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:13:04.306030 27729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:13:04.311255 27729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:44741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:44741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=37867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:04.312311 27729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:04.313303 27729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:04.321924 27734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:04.321939 27735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:04.321935 27737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:04.322005 27729 server_base.cc:1034] running on GCE node
I20260806 02:13:04.322651 27729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:04.323800 27729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:04.324956 27729 hybrid_clock.cc:648] HybridClock initialized: now 1785982384324938 us; error 29 us; skew 500 ppm
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982384, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:13:04.327672 27729 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:13:04.328577 27729 webserver.cc:460] Webserver started at http://127.18.70.126:37867/ using document root <none> and password file <none>
I20260806 02:13:04.329129 27729 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:04.329265 27729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:04.333021 27729 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 02:13:04.335055 27745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:04.336298 27729 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:13:04.336440 27729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "8696c0bcd6d4490a80c448af6e3b0d7c"
format_stamp: "Formatted at 2026-08-06 02:13:03 on dist-test-slave-6k22"
server_key: "e5a93b547d0dfcc8483e2a7e030f3604"
server_key_iv: "ee1ec1b1050eca754a590ce03a9d3b96"
server_key_version: "encryptionkey@0"
I20260806 02:13:04.336962 27729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:04.348609 27729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:04.351331 27729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:04.351529 27729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:04.358716 27729 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:44741
I20260806 02:13:04.358772 27799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:44741 every 8 connection(s)
I20260806 02:13:04.359894 27729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:13:04.364241 27800 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:13:04.365847 27800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c: Bootstrap starting.
I20260806 02:13:04.368547 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 27729
I20260806 02:13:04.369072 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:43333
--local_ip_for_outbound_sockets=127.18.70.65
--tserver_master_addrs=127.18.70.126:44741
--webserver_port=40691
--webserver_interface=127.18.70.65
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:37955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.70.65
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:13:04.383926 27800 log.cc:824] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c: Log is configured to *not* fsync() on all Append() calls
I20260806 02:13:04.390518 27800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:13:04.390910 27800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c: Bootstrap complete.
I20260806 02:13:04.395438 27800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44741 } }
I20260806 02:13:04.395918 27800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8696c0bcd6d4490a80c448af6e3b0d7c, State: Initialized, Role: FOLLOWER
I20260806 02:13:04.396416 27800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44741 } }
I20260806 02:13:04.396534 27800 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:13:04.396624 27800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:13:04.396730 27800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 1 FOLLOWER]: Advancing to term 2
I20260806 02:13:04.398649 27800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44741 } }
I20260806 02:13:04.398993 27800 leader_election.cc:302] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8696c0bcd6d4490a80c448af6e3b0d7c; no voters:
I20260806 02:13:04.399331 27800 leader_election.cc:288] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 02:13:04.399413 27804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 2 FOLLOWER]: Leader election won for term 2
I20260806 02:13:04.399696 27804 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [term 2 LEADER]: Becoming Leader. State: Replica: 8696c0bcd6d4490a80c448af6e3b0d7c, State: Running, Role: LEADER
I20260806 02:13:04.400048 27804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44741 } }
I20260806 02:13:04.400632 27800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:13:04.401062 27804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8696c0bcd6d4490a80c448af6e3b0d7c. Latest consensus state: current_term: 2 leader_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44741 } } }
I20260806 02:13:04.401190 27804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [sys.catalog]: This master's current role is: LEADER
I20260806 02:13:04.401590 27808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:13:04.401718 27805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8696c0bcd6d4490a80c448af6e3b0d7c" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 44741 } } }
I20260806 02:13:04.401909 27805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c [sys.catalog]: This master's current role is: LEADER
I20260806 02:13:04.405146 27808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:13:04.405891 27808 catalog_manager.cc:1259] Loaded cluster ID: 446f0c2d2cea49498a69e9760581dff2
I20260806 02:13:04.405967 27808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:13:04.407728 27808 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:13:04.408298 27808 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8696c0bcd6d4490a80c448af6e3b0d7c: Loaded TSK: 0
I20260806 02:13:04.409359 27808 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 02:13:04.495783 27802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:04.496057 27802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:04.496129 27802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:04.500151 27802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:04.500253 27802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:04.500350 27802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:13:04.506193 27802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.65.keytab
--principal=oryx/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:43333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=40691
--tserver_master_addrs=127.18.70.126:44741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:04.507229 27802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:04.508285 27802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:04.518296 27828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:04.518260 27829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:04.518548 27831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:04.518692 27802 server_base.cc:1034] running on GCE node
I20260806 02:13:04.519112 27802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:04.519845 27802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:04.521036 27802 hybrid_clock.cc:648] HybridClock initialized: now 1785982384521011 us; error 44 us; skew 500 ppm
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982384, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:13:04.528815 27802 init.cc:377] Logged in from keytab as oryx/127.18.70.65@KRBTEST.COM (short username oryx)
I20260806 02:13:04.529832 27802 webserver.cc:460] Webserver started at http://127.18.70.65:40691/ using document root <none> and password file <none>
I20260806 02:13:04.530376 27802 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:04.530452 27802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:04.534462 27802 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 02:13:04.536646 27838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:04.537940 27802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:13:04.538033 27802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "49cc7e3141cd4c7fa20e1b459996af81"
format_stamp: "Formatted at 2026-08-06 02:13:03 on dist-test-slave-6k22"
server_key: "09c59033acedea795d65eb4533e57bd1"
server_key_iv: "b58b5073ed9e681a525183cff6504aef"
server_key_version: "encryptionkey@0"
I20260806 02:13:04.538551 27802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:04.552132 27802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:04.555320 27802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:04.555480 27802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:04.556104 27802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:04.556993 27802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:04.557037 27802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:04.557075 27802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:04.557089 27802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:04.566206 27802 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:43333
I20260806 02:13:04.566234 27958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:43333 every 8 connection(s)
I20260806 02:13:04.567327 27802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:13:04.577000 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 27802
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:04.577595 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:33133
--local_ip_for_outbound_sockets=127.18.70.66
--tserver_master_addrs=127.18.70.126:44741
--webserver_port=42189
--webserver_interface=127.18.70.66
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:37955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.70.66
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:04.580003 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:04.569593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:37003)
0806 02:13:04.570306 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:04.570348 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:04.571156 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:04.572236 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:04.572243 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:04.572630 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:04.573090 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.573103 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.574316 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.574319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.575140 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.575149 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.575935 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.575938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.576071 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.576613 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:04.576629 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:04.579191 (+ 2562us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":187,"threads_started":1}
W20260806 02:13:04.580494 27959 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:44741 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.70.126:44741: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
W20260806 02:13:04.580560 27961 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:04.569467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:37003 (local address 127.18.70.126:44741)
0806 02:13:04.569936 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:04.569957 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:04.570487 (+ 530us) server_negotiation.cc:407] Connection header received
0806 02:13:04.571506 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:04.571517 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:04.571877 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:04.572144 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:04.573238 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.574202 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.575242 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.575798 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.575918 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.579679 (+ 3761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:37003: BlockingRecv error: recv got EOF from 127.18.70.65:37003 (error 108)
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":106,"threads_started":1}
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:04.586766 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:04.580950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:58809)
0806 02:13:04.581160 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:04.581175 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:04.581247 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:04.581500 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:04.581502 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:04.581770 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:04.581862 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.581867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.582953 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.582955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.583474 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.583481 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.584142 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.584145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.584216 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.584731 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:04.584749 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:04.586581 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":22}
W20260806 02:13:04.586798 27961 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:04.581019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:58809 (local address 127.18.70.126:44741)
0806 02:13:04.581261 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:04.581266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:04.581290 (+ 24us) server_negotiation.cc:407] Connection header received
0806 02:13:04.581331 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:04.581335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:04.581387 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:04.581481 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:04.581982 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.582842 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.583612 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.584021 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.584156 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.586699 (+ 2543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:58809: BlockingRecv error: recv got EOF from 127.18.70.65:58809 (error 108)
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":162}
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:04.592270 27962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:04.587335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:44943)
0806 02:13:04.587530 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:04.587545 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:04.587635 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:04.587890 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:04.587893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:04.588040 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:04.588104 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.588108 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.588829 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.588834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.589340 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.589348 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.589827 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.589830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.589911 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.590414 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:04.590425 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:04.592134 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112}
W20260806 02:13:04.592381 27961 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:04.587453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:44943 (local address 127.18.70.126:44741)
0806 02:13:04.587609 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:04.587613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:04.587626 (+ 13us) server_negotiation.cc:407] Connection header received
0806 02:13:04.587750 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:04.587756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:04.587793 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:04.587874 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:04.588215 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.588712 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.589467 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.589673 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.589803 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.592258 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:44943: BlockingRecv error: recv got EOF from 127.18.70.65:44943 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 02:13:04.592609 27959 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 02:13:04.707722 27963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:04.707976 27963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:04.708019 27963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:04.712011 27963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:04.712064 27963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:04.712121 27963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:13:04.717216 27963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.66.keytab
--principal=oryx/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:33133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=42189
--tserver_master_addrs=127.18.70.126:44741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:04.718191 27963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:04.719184 27963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:04.729185 27968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:04.729192 27972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:04.729207 27969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:04.729362 27963 server_base.cc:1034] running on GCE node
I20260806 02:13:04.729892 27963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:04.730590 27963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:04.731793 27963 hybrid_clock.cc:648] HybridClock initialized: now 1785982384731760 us; error 43 us; skew 500 ppm
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982384, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:13:04.734421 27963 init.cc:377] Logged in from keytab as oryx/127.18.70.66@KRBTEST.COM (short username oryx)
I20260806 02:13:04.735250 27963 webserver.cc:460] Webserver started at http://127.18.70.66:42189/ using document root <none> and password file <none>
I20260806 02:13:04.735941 27963 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:04.736047 27963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:04.739903 27963 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 02:13:04.742015 27979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:04.743326 27963 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:13:04.743455 27963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "1a018cbf1f5a41b988f59c783d4f2303"
format_stamp: "Formatted at 2026-08-06 02:13:03 on dist-test-slave-6k22"
server_key: "9685bfd69c693ebe6d8bfb29c83e8484"
server_key_iv: "277db0d21c1829f005a48babd04ea3a9"
server_key_version: "encryptionkey@0"
I20260806 02:13:04.744063 27963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:04.766037 27963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:04.768846 27963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:04.769045 27963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:04.769634 27963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:04.770426 27963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:04.770496 27963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:04.770557 27963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:04.770605 27963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:04.779788 27963 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:33133
I20260806 02:13:04.779827 28096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:33133 every 8 connection(s)
I20260806 02:13:04.780778 27963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:13:04.786375 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 27963
I20260806 02:13:04.786824 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:45811
--local_ip_for_outbound_sockets=127.18.70.67
--tserver_master_addrs=127.18.70.126:44741
--webserver_port=41145
--webserver_interface=127.18.70.67
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:37955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.70.67
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 02:13:04.791546 27961 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:04.782588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:47501 (local address 127.18.70.126:44741)
0806 02:13:04.782763 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:04.782768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:04.783859 (+ 1091us) server_negotiation.cc:407] Connection header received
0806 02:13:04.784670 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:04.784672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:04.784723 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:04.784799 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:04.785899 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.786410 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.787673 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.787906 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.788027 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.791426 (+ 3399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:47501: BlockingRecv error: recv got EOF from 127.18.70.66:47501 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 02:13:04.792135 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:04.782980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:47501)
0806 02:13:04.783711 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:04.783752 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:04.784491 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:04.784968 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:04.784974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:04.785312 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:04.785765 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.785780 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.786556 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.786560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.787515 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.787526 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.788069 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.788074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.788240 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.788856 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:04.788873 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:04.791292 (+ 2419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":115,"threads_started":1}
W20260806 02:13:04.792563 28097 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:44741 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.70.126:44741: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:04.798918 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:04.793023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:40239)
0806 02:13:04.793263 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:04.793279 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:04.793373 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:04.793623 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:04.793627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:04.793820 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:04.793965 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.793973 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.794660 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.794665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.795719 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.795733 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.796164 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.796171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.796292 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.796728 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:04.796748 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:04.798743 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157}
W20260806 02:13:04.798934 27961 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:04.793098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:40239 (local address 127.18.70.126:44741)
0806 02:13:04.793213 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:04.793216 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:04.793342 (+ 126us) server_negotiation.cc:407] Connection header received
0806 02:13:04.793458 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:04.793460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:04.793493 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:04.793555 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:04.794099 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.794552 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.795849 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.796046 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.796122 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.798853 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:40239: BlockingRecv error: recv got EOF from 127.18.70.66:40239 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:04.804843 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:04.799658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:57935)
0806 02:13:04.799814 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:04.799826 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:04.799905 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:04.800130 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:04.800133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:04.800288 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:04.800370 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.800376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.801133 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.801136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.802032 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.802041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.802471 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.802475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.802570 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.802967 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:04.802980 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:04.804725 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260806 02:13:04.804946 27961 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:04.799722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:57935 (local address 127.18.70.126:44741)
0806 02:13:04.799837 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:04.799840 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:04.799869 (+ 29us) server_negotiation.cc:407] Connection header received
0806 02:13:04.799991 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:04.799993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:04.800026 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:04.800090 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:04.800472 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.800992 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.802135 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.802369 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.802503 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.804841 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:57935: BlockingRecv error: recv got EOF from 127.18.70.66:57935 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 02:13:04.805203 28097 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 02:13:04.905047 28100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:04.905303 28100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:04.905381 28100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:04.909555 28100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:04.909626 28100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:04.909735 28100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:13:04.915117 28100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/krb5kdc/oryx_127.18.70.67.keytab
--principal=oryx/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:45811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=41145
--tserver_master_addrs=127.18.70.126:44741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:04.916205 28100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:04.917227 28100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:04.926855 28105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:04.926855 28106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:04.926890 28108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:04.927130 28100 server_base.cc:1034] running on GCE node
I20260806 02:13:04.927492 28100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:04.928197 28100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:04.929355 28100 hybrid_clock.cc:648] HybridClock initialized: now 1785982384929342 us; error 27 us; skew 500 ppm
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982384, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:13:04.932049 28100 init.cc:377] Logged in from keytab as oryx/127.18.70.67@KRBTEST.COM (short username oryx)
I20260806 02:13:04.932998 28100 webserver.cc:460] Webserver started at http://127.18.70.67:41145/ using document root <none> and password file <none>
I20260806 02:13:04.933576 28100 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:04.933670 28100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:04.937458 28100 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 02:13:04.939500 28115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:04.940812 28100 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:13:04.940948 28100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "63412137813148ce93fa162de656b354"
format_stamp: "Formatted at 2026-08-06 02:13:04 on dist-test-slave-6k22"
server_key: "9a4f70fd7e66d50f450b5911c15cd6a2"
server_key_iv: "8242ddb78b80eec02358258f4445500f"
server_key_version: "encryptionkey@0"
I20260806 02:13:04.941486 28100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:04.961978 28100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:04.964870 28100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:04.965102 28100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:04.965724 28100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:04.966555 28100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:04.966637 28100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:04.966708 28100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:04.966743 28100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:04.975826 28100 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:45811
I20260806 02:13:04.975848 28232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:45811 every 8 connection(s)
I20260806 02:13:04.976874 28100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:13:04.984200 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 28100
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 02:13:04.988095 27961 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:04.978681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:55201 (local address 127.18.70.126:44741)
0806 02:13:04.978829 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:04.978832 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:04.979900 (+ 1068us) server_negotiation.cc:407] Connection header received
0806 02:13:04.980715 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:04.980717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:04.980763 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:04.980842 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:04.981985 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.982513 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.983691 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.983926 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.984006 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.987964 (+ 3958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:55201: BlockingRecv error: recv got EOF from 127.18.70.67:55201 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:13:04.988801 28235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:04.979009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:55201)
0806 02:13:04.979720 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:04.979771 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:04.980543 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:04.981010 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:04.981016 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:04.981353 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:04.981848 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.981862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.982634 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.982637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.983509 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.983518 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.984086 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.984090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.984299 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.984999 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:04.985019 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:04.987871 (+ 2852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":108,"threads_started":1}
W20260806 02:13:04.989326 28233 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:44741 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.70.126:44741: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:04.994712 28235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:04.989778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:34949)
0806 02:13:04.989868 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:04.989883 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:04.989953 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:04.990201 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:04.990204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:04.990360 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:04.990442 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.990447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.991290 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.991294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.991871 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.991880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.992301 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.992305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.992391 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.992833 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:04.992844 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:04.994591 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 02:13:04.995010 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:04.985384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48842 (local address 127.18.70.126:44741)
0806 02:13:04.985704 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:04.985709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:04.985722 (+ 13us) server_negotiation.cc:407] Connection header received
0806 02:13:04.985764 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:04.985765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:04.985801 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:04.985876 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:04.986804 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.987641 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.988639 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.988963 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.989070 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.990541 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:04.990553 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:04.990567 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:13:04.990592 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:04.992264 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:04.992705 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:04.992712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:04.992718 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:04.992838 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:04.993160 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:04.993163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:04.993165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:04.993663 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:04.993865 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:04.994592 (+ 727us) server_negotiation.cc:299] Negotiation successful
0806 02:13:04.994842 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":127,"threads_started":1}
W20260806 02:13:04.996623 27961 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:04.989874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:34949 (local address 127.18.70.126:44741)
0806 02:13:04.990023 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:04.990027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:04.990038 (+ 11us) server_negotiation.cc:407] Connection header received
0806 02:13:04.990072 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:04.990073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:04.990103 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:04.990185 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:04.990557 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.991178 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.991988 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.992195 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.992301 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:04.996516 (+ 4215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:34949: BlockingRecv error: recv got EOF from 127.18.70.67:34949 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:04 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:05.002604 28235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:04.996981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:41307)
0806 02:13:04.997953 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:04.997972 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:04.998055 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:04.998284 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:04.998286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:04.998417 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:04.998484 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.998488 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.999228 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.999232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:04.999818 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:04.999826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:05.000341 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:05.000344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:05.000400 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:05.000765 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:05.000784 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:05.002496 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 02:13:05.002750 27961 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:04.997132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:41307 (local address 127.18.70.126:44741)
0806 02:13:04.997266 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:04.997272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:04.998045 (+ 773us) server_negotiation.cc:407] Connection header received
0806 02:13:04.998128 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:04.998130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:04.998171 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:04.998248 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:04.998572 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:04.999074 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:04.999916 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:05.000125 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:05.000331 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:05.002624 (+ 2293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:41307: BlockingRecv error: recv got EOF from 127.18.70.67:41307 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 02:13:05.002842 28233 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 02:13:05 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:05 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:05.598449 28238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:05.593259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:40849)
0806 02:13:05.593569 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:05.593585 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:05.593680 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:05.593955 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:05.593957 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:05.594140 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:05.594236 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:05.594241 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:05.594989 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:05.594991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:05.595612 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:05.595621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:05.596016 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:05.596019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:05.596078 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:05.596557 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:05.596568 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:05.598312 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":97,"threads_started":1}
W20260806 02:13:05.598709 28239 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:05.593461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:40849 (local address 127.18.70.126:44741)
0806 02:13:05.593745 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:05.593750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:05.593762 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:13:05.593798 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:05.593800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:05.593840 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:05.593915 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:05.594351 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:05.594894 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:05.595707 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:05.595920 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:05.595996 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:05.598452 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:40849: BlockingRecv error: recv got EOF from 127.18.70.65:40849 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":138,"threads_started":1}
Aug 06 02:13:05 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:05 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:05.811771 28240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:05.805861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:33671)
0806 02:13:05.806146 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:05.806164 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:05.806308 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:05.806586 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:05.806589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:05.806812 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:05.806924 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:05.806930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:05.807753 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:05.807758 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:05.808543 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:05.808553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:05.809197 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:05.809200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:05.809288 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:05.809758 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:05.809782 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:05.811615 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":82,"threads_started":1}
W20260806 02:13:05.811887 28239 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:05.805913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:33671 (local address 127.18.70.126:44741)
0806 02:13:05.806088 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:05.806093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:05.806246 (+ 153us) server_negotiation.cc:407] Connection header received
0806 02:13:05.806420 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:05.806424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:05.806475 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:05.806565 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:05.807049 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:05.807646 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:05.808698 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:05.809077 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:05.809185 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:05.811777 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:33671: BlockingRecv error: recv got EOF from 127.18.70.66:33671 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 02:13:06 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:06 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:06.009439 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:06.003500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:51447)
0806 02:13:06.003805 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:06.003820 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:06.003959 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:06.004238 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:06.004241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:06.004488 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:06.004639 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:06.004648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:06.005684 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:06.005688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:06.006347 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:06.006357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:06.006784 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:06.006786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:06.006859 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:06.007322 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:06.007333 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:06.009249 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":121,"threads_started":1}
W20260806 02:13:06.009467 28239 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:06.003562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:51447 (local address 127.18.70.126:44741)
0806 02:13:06.003744 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:06.003749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:06.003904 (+ 155us) server_negotiation.cc:407] Connection header received
0806 02:13:06.004068 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:06.004072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:06.004128 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:06.004219 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:06.004802 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:06.005565 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:06.006457 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:06.006674 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:06.006765 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:06.009364 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:51447: BlockingRecv error: recv got EOF from 127.18.70.67:51447 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 02:13:06 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:06 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:06.604679 28242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:06.599194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:48165)
0806 02:13:06.599484 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:06.599508 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:06.599669 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:06.599913 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:06.599915 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:06.600090 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:06.600267 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:06.600277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:06.601007 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:06.601010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:06.601660 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:06.601669 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:06.602085 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:06.602088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:06.602148 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:06.602700 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:06.602713 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:06.604539 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":74,"threads_started":1}
W20260806 02:13:06.604830 28244 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:06.599331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:48165 (local address 127.18.70.126:44741)
0806 02:13:06.599597 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:06.599600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:06.599614 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:13:06.599754 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:06.599756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:06.599800 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:06.599874 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:06.600378 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:06.600913 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:06.601760 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:06.601975 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:06.602089 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:06.604667 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:48165: BlockingRecv error: recv got EOF from 127.18.70.65:48165 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
Aug 06 02:13:06 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:06 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:06.818336 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:06.812785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:49907)
0806 02:13:06.813054 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:06.813067 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:06.813166 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:06.813372 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:06.813374 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:06.813548 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:06.813646 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:06.813651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:06.814490 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:06.814493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:06.815065 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:06.815073 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:06.815460 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:06.815463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:06.815524 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:06.816142 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:06.816155 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:06.818215 (+ 2060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":97,"threads_started":1}
W20260806 02:13:06.818531 28244 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:06.812822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:49907 (local address 127.18.70.126:44741)
0806 02:13:06.812974 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:06.812978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:06.813137 (+ 159us) server_negotiation.cc:407] Connection header received
0806 02:13:06.813245 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:06.813247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:06.813290 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:06.813360 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:06.813742 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:06.814389 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:06.815164 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:06.815364 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:06.815435 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:06.818343 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:49907: BlockingRecv error: recv got EOF from 127.18.70.66:49907 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 02:13:07 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:07 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:07.021227 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:07.010377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:56179)
0806 02:13:07.010694 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:07.010709 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:07.010829 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:07.011084 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:07.011087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:07.011284 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:07.011376 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:07.011381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:07.012198 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:07.012201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:07.013046 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:07.013058 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:07.013490 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:07.013493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:07.013549 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:07.019227 (+ 5678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:07.019240 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:07.021102 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":117,"threads_started":1}
W20260806 02:13:07.021389 28244 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:07.010561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:56179 (local address 127.18.70.126:44741)
0806 02:13:07.010694 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:07.010699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:07.010763 (+ 64us) server_negotiation.cc:407] Connection header received
0806 02:13:07.010928 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:07.010930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:07.010981 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:07.011061 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:07.011484 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:07.012078 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:07.013163 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:07.013394 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:07.013470 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:07.021235 (+ 7765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:56179: BlockingRecv error: recv got EOF from 127.18.70.67:56179 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 02:13:07 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:07 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:07.610764 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:07.605537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:43033)
0806 02:13:07.605812 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:07.605827 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:07.605930 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:07.606182 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:07.606185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:07.606377 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:07.606480 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:07.606485 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:07.607210 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:07.607213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:07.607796 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:07.607803 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:07.608222 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:07.608225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:07.608294 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:07.608784 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:07.608796 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:07.610634 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":96,"threads_started":1}
W20260806 02:13:07.610908 28249 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:07.605586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:43033 (local address 127.18.70.126:44741)
0806 02:13:07.605850 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:07.605854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:07.605882 (+ 28us) server_negotiation.cc:407] Connection header received
0806 02:13:07.606016 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:07.606019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:07.606064 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:07.606142 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:07.606581 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:07.607116 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:07.607899 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:07.608120 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:07.608202 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:07.610768 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:43033: BlockingRecv error: recv got EOF from 127.18.70.65:43033 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":106,"threads_started":1}
Aug 06 02:13:07 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:07 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:07.824595 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:07.819188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:40581)
0806 02:13:07.819511 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:07.819528 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:07.819698 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:07.820006 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:07.820008 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:07.820186 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:07.820316 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:07.820324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:07.821055 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:07.821058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:07.821626 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:07.821633 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:07.822043 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:07.822046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:07.822107 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:07.822590 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:07.822602 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:07.824469 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":131,"threads_started":1}
W20260806 02:13:07.824772 28249 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:07.819248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:40581 (local address 127.18.70.126:44741)
0806 02:13:07.819411 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:07.819416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:07.819636 (+ 220us) server_negotiation.cc:407] Connection header received
0806 02:13:07.819805 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:07.819809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:07.819870 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:07.819948 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:07.820433 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:07.820944 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:07.821727 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:07.821946 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:07.822020 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:07.824601 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:40581: BlockingRecv error: recv got EOF from 127.18.70.66:40581 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 02:13:08 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:08 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:08.027407 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:08.022010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:49225)
0806 02:13:08.022316 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:08.022330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:08.022431 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:08.022670 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:08.022672 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:08.022853 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:08.022952 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:08.022958 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:08.023721 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:08.023723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:08.024287 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:08.024295 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:08.024702 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:08.024705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:08.024764 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:08.025372 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:08.025384 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:08.027280 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":122,"threads_started":1}
W20260806 02:13:08.027518 28249 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:08.022060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:49225 (local address 127.18.70.126:44741)
0806 02:13:08.022209 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:08.022214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:08.022402 (+ 188us) server_negotiation.cc:407] Connection header received
0806 02:13:08.022509 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:08.022511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:08.022562 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:08.022640 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:08.023048 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:08.023620 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:08.024391 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:08.024603 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:08.024682 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:08.027404 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:49225: BlockingRecv error: recv got EOF from 127.18.70.67:49225 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 02:13:08 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:08 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:08.617292 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:08.611784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:38865)
0806 02:13:08.612102 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:08.612116 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:08.612212 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:08.612479 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:08.612481 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:08.612661 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:08.612760 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:08.612765 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:08.613494 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:08.613496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:08.614042 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:08.614048 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:08.614446 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:08.614448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:08.614508 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:08.614993 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:08.615005 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:08.617110 (+ 2105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":120,"threads_started":1}
W20260806 02:13:08.617347 28253 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:08.611828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:38865 (local address 127.18.70.126:44741)
0806 02:13:08.612121 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:08.612124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:08.612165 (+ 41us) server_negotiation.cc:407] Connection header received
0806 02:13:08.612314 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:08.612316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:08.612360 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:08.612437 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:08.612871 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:08.613402 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:08.614139 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:08.614352 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:08.614425 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:08.617230 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:38865: BlockingRecv error: recv got EOF from 127.18.70.65:38865 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":94,"threads_started":1}
Aug 06 02:13:08 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:08 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:08.831355 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:08.825441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:56477)
0806 02:13:08.825811 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:08.825824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:08.825958 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:08.826247 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:08.826251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:08.826471 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:08.826570 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:08.826576 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:08.827388 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:08.827394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:08.828055 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:08.828063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:08.828548 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:08.828552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:08.828644 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:08.829215 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:08.829228 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:08.831180 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":91,"threads_started":1}
W20260806 02:13:08.831444 28253 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:08.825505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:56477 (local address 127.18.70.126:44741)
0806 02:13:08.825686 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:08.825690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:08.825908 (+ 218us) server_negotiation.cc:407] Connection header received
0806 02:13:08.826069 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:08.826074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:08.826126 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:08.826219 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:08.826700 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:08.827238 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:08.828184 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:08.828426 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:08.828543 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:08.831320 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:56477: BlockingRecv error: recv got EOF from 127.18.70.66:56477 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 02:13:09 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:09 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:09.033382 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:09.028249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:43881)
0806 02:13:09.028585 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:09.028599 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:09.028711 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:09.028937 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:09.028939 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:09.029114 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:09.029222 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:09.029227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:09.029913 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:09.029915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:09.030444 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:09.030450 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:09.030831 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:09.030833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:09.030893 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:09.031363 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:09.031373 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:09.033215 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":119,"threads_started":1}
W20260806 02:13:09.033470 28253 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:09.028420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:43881 (local address 127.18.70.126:44741)
0806 02:13:09.028575 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:09.028579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:09.028665 (+ 86us) server_negotiation.cc:407] Connection header received
0806 02:13:09.028780 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:09.028782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:09.028824 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:09.028896 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:09.029320 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:09.029815 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:09.030536 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:09.030738 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:09.030810 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:09.033352 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:43881: BlockingRecv error: recv got EOF from 127.18.70.67:43881 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 02:13:09 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:09 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:09.623962 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:09.618081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:45943)
0806 02:13:09.618427 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:09.618441 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:09.618530 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:09.618778 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:09.618781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:09.618964 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:09.619057 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:09.619063 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:09.619821 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:09.619829 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:09.620671 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:09.620681 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:09.621213 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:09.621217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:09.621312 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:09.621908 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:09.621920 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:09.623789 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":115,"threads_started":1}
W20260806 02:13:09.624043 28257 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:09.618229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:45943 (local address 127.18.70.126:44741)
0806 02:13:09.618493 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:09.618497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:09.618510 (+ 13us) server_negotiation.cc:407] Connection header received
0806 02:13:09.618614 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:09.618617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:09.618660 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:09.618736 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:09.619156 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:09.619717 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:09.620793 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:09.621096 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:09.621211 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:09.623918 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:45943: BlockingRecv error: recv got EOF from 127.18.70.65:45943 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":60,"threads_started":1}
Aug 06 02:13:09 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:09 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:09.838217 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:09.832437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:54083)
0806 02:13:09.832755 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:09.832773 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:09.832895 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:09.833156 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:09.833159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:09.833358 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:09.833479 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:09.833486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:09.834214 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:09.834217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:09.834803 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:09.834812 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:09.835221 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:09.835224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:09.835294 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:09.836024 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:09.836049 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:09.838043 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":142,"threads_started":1}
W20260806 02:13:09.838292 28257 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:09.832574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:54083 (local address 127.18.70.126:44741)
0806 02:13:09.832749 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:09.832753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:09.832866 (+ 113us) server_negotiation.cc:407] Connection header received
0806 02:13:09.832982 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:09.832984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:09.833025 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:09.833101 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:09.833589 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:09.834096 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:09.834913 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:09.835117 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:09.835188 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:09.838168 (+ 2980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:54083: BlockingRecv error: recv got EOF from 127.18.70.66:54083 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 06 02:13:10 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:10 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:10.040364 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:10.034340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:35505)
0806 02:13:10.034631 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:10.034650 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:10.034782 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:10.035098 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:10.035101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:10.035327 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:10.035426 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:10.035434 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:10.036399 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:10.036403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:10.036981 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:10.036990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:10.037473 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:10.037477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:10.037602 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:10.038089 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:10.038100 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:10.040246 (+ 2146us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":104,"threads_started":1}
W20260806 02:13:10.040670 28257 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:10.034382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:35505 (local address 127.18.70.126:44741)
0806 02:13:10.034533 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:10.034537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:10.034751 (+ 214us) server_negotiation.cc:407] Connection header received
0806 02:13:10.034881 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:10.034885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:10.034947 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:10.035057 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:10.035581 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:10.036278 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:10.037114 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:10.037353 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:10.037457 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:10.040432 (+ 2975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:35505: BlockingRecv error: recv got EOF from 127.18.70.67:35505 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 02:13:10 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:10 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:10.630256 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:10.624729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:56395)
0806 02:13:10.625015 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:10.625029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:10.625154 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:10.625472 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:10.625475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:10.625662 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:10.625762 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:10.625769 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:10.626580 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:10.626583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:10.627138 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:10.627145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:10.627657 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:10.627660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:10.627724 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:10.628312 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:10.628324 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:10.630129 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":92,"threads_started":1}
W20260806 02:13:10.630438 28261 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:10.624801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:56395 (local address 127.18.70.126:44741)
0806 02:13:10.625060 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:10.625065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:10.625094 (+ 29us) server_negotiation.cc:407] Connection header received
0806 02:13:10.625257 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:10.625263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:10.625327 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:10.625419 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:10.625919 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:10.626480 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:10.627236 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:10.627481 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:10.627629 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:10.630255 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:56395: BlockingRecv error: recv got EOF from 127.18.70.65:56395 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":96,"threads_started":1}
Aug 06 02:13:10 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:10 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:10.845414 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:10.839178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:54233)
0806 02:13:10.839505 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:10.839519 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:10.839651 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:10.839902 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:10.839904 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:10.840165 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:10.840265 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:10.840282 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:10.841288 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:10.841292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:10.842008 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:10.842019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:10.842532 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:10.842565 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:10.842663 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:10.843191 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:10.843203 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:10.845188 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":141,"threads_started":1}
W20260806 02:13:10.845467 28261 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:10.839246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:54233 (local address 127.18.70.126:44741)
0806 02:13:10.839436 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:10.839441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:10.839612 (+ 171us) server_negotiation.cc:407] Connection header received
0806 02:13:10.839735 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:10.839737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:10.839795 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:10.839867 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:10.840423 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:10.841176 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:10.842148 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:10.842428 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:10.842512 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:10.845325 (+ 2813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:54233: BlockingRecv error: recv got EOF from 127.18.70.66:54233 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 02:13:11 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:11 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:11.046954 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:11.041168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:44195)
0806 02:13:11.041448 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:11.041462 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:11.041608 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:11.041894 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:11.041896 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:11.042144 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:11.042246 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:11.042252 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:11.043047 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:11.043050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:11.043787 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:11.043797 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:11.044192 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:11.044195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:11.044257 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:11.044741 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:11.044759 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:11.046781 (+ 2022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":103,"threads_started":1}
W20260806 02:13:11.047199 28261 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:11.041204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:44195 (local address 127.18.70.126:44741)
0806 02:13:11.041345 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:11.041349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:11.041534 (+ 185us) server_negotiation.cc:407] Connection header received
0806 02:13:11.041730 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:11.041732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:11.041783 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:11.041852 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:11.042359 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:11.042936 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:11.043887 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:11.044097 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:11.044175 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:11.046980 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:44195: BlockingRecv error: recv got EOF from 127.18.70.67:44195 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 02:13:11 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:11 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:11.636277 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:11.631030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:46261)
0806 02:13:11.631385 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:11.631399 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:11.631501 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:11.631771 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:11.631774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:11.631964 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:11.632063 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:11.632069 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:11.632795 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:11.632798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:11.633340 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:11.633347 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:11.633736 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:11.633738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:11.633809 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:11.634304 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:11.634316 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:11.636149 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":87,"threads_started":1}
W20260806 02:13:11.636598 28265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:11.631151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:46261 (local address 127.18.70.126:44741)
0806 02:13:11.631390 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:11.631394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:11.631476 (+ 82us) server_negotiation.cc:407] Connection header received
0806 02:13:11.631596 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:11.631599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:11.631644 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:11.631718 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:11.632165 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:11.632692 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:11.633437 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:11.633643 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:11.633714 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:11.636331 (+ 2617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:46261: BlockingRecv error: recv got EOF from 127.18.70.65:46261 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":65,"threads_started":1}
Aug 06 02:13:11 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:11 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:11.851806 28266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:11.846324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:50603)
0806 02:13:11.846682 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:11.846697 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:11.846804 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:11.847043 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:11.847045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:11.847237 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:11.847334 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:11.847340 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:11.848197 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:11.848200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:11.848866 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:11.848876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:11.849277 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:11.849279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:11.849344 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:11.849822 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:11.849834 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:11.851692 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":93,"threads_started":1}
W20260806 02:13:11.851878 28265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:11.846344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:50603 (local address 127.18.70.126:44741)
0806 02:13:11.846495 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:11.846499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:11.846778 (+ 279us) server_negotiation.cc:407] Connection header received
0806 02:13:11.846884 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:11.846886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:11.846932 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:11.847001 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:11.847435 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:11.848087 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:11.848967 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:11.849179 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:11.849257 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:11.851788 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:50603: BlockingRecv error: recv got EOF from 127.18.70.66:50603 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 02:13:12 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:12 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:12.058086 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:12.047754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:49143)
0806 02:13:12.048075 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:12.048088 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:12.048225 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:12.048516 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:12.048519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:12.048816 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:12.048925 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:12.048931 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:12.049693 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:12.049698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:12.050267 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:12.050276 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:12.050741 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:12.050745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:12.050864 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:12.056055 (+ 5191us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:12.056067 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:12.057913 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":73,"threads_started":1}
W20260806 02:13:12.058147 28265 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:12.047799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:49143 (local address 127.18.70.126:44741)
0806 02:13:12.047964 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:12.047969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:12.048160 (+ 191us) server_negotiation.cc:407] Connection header received
0806 02:13:12.048333 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:12.048336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:12.048397 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:12.048493 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:12.049046 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:12.049581 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:12.050399 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:12.050624 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:12.050727 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:12.058041 (+ 7314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:49143: BlockingRecv error: recv got EOF from 127.18.70.67:49143 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 02:13:12 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:12 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:12.643034 28268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:12.637260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:51347)
0806 02:13:12.637724 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:12.637740 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:12.637865 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:12.638143 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:12.638145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:12.638326 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:12.638437 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:12.638442 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:12.639247 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:12.639250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:12.639846 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:12.639854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:12.640386 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:12.640389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:12.640449 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:12.641003 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:12.641014 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:12.642913 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":99,"threads_started":1}
W20260806 02:13:12.643225 28269 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:12.637410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:51347 (local address 127.18.70.126:44741)
0806 02:13:12.637676 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:12.637683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:12.637827 (+ 144us) server_negotiation.cc:407] Connection header received
0806 02:13:12.637963 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:12.637967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:12.638023 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:12.638107 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:12.638551 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:12.639144 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:12.639976 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:12.640206 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:12.640378 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:12.643049 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:51347: BlockingRecv error: recv got EOF from 127.18.70.65:51347 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":68,"threads_started":1}
Aug 06 02:13:12 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:12 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:12.858671 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:12.852654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:58597)
0806 02:13:12.852963 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:12.852977 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:12.853083 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:12.853321 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:12.853324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:12.853504 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:12.853602 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:12.853607 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:12.854596 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:12.854599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:12.855246 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:12.855255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:12.855829 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:12.855833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:12.855899 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:12.856515 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:12.856528 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:12.858487 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":100,"threads_started":1}
W20260806 02:13:12.858717 28269 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:12.852722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:58597 (local address 127.18.70.126:44741)
0806 02:13:12.852883 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:12.852890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:12.853052 (+ 162us) server_negotiation.cc:407] Connection header received
0806 02:13:12.853151 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:12.853153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:12.853198 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:12.853264 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:12.853702 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:12.854470 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:12.855396 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:12.855698 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:12.855781 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:12.858615 (+ 2834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:58597: BlockingRecv error: recv got EOF from 127.18.70.66:58597 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 02:13:13 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:13 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:13.064162 28271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:13.058973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:48911)
0806 02:13:13.059274 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:13.059289 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:13.059393 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:13.059639 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:13.059642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:13.059832 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:13.059929 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:13.059934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:13.060661 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:13.060663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:13.061209 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:13.061216 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:13.061636 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:13.061640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:13.061736 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:13.062277 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:13.062289 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:13.064051 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":103,"threads_started":1}
W20260806 02:13:13.064401 28269 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:13.059015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:48911 (local address 127.18.70.126:44741)
0806 02:13:13.059170 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:13.059175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:13.059361 (+ 186us) server_negotiation.cc:407] Connection header received
0806 02:13:13.059459 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:13.059461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:13.059509 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:13.059621 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:13.060025 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:13.060547 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:13.061302 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:13.061513 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:13.061622 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:13.064178 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:48911: BlockingRecv error: recv got EOF from 127.18.70.67:48911 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 02:13:13 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:13 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:13.650065 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:13.643904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:36619)
0806 02:13:13.644184 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:13.644197 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:13.644303 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:13.644550 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:13.644552 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:13.644789 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:13.644910 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:13.644921 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:13.645699 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:13.645719 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:13.646338 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:13.646347 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:13.647022 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:13.647029 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:13.647164 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:13.647829 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:13.647885 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:13.649907 (+ 2022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":73,"threads_started":1}
W20260806 02:13:13.650173 28273 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:13.643948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:36619 (local address 127.18.70.126:44741)
0806 02:13:13.644234 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:13.644238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:13.644252 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:13:13.644387 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:13.644389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:13.644434 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:13.644513 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:13.645028 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:13.645602 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:13.646437 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:13.646652 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:13.646789 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:13.650049 (+ 3260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:36619: BlockingRecv error: recv got EOF from 127.18.70.65:36619 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":97,"threads_started":1}
Aug 06 02:13:13 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:13 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:13.864786 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:13.859495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:56381)
0806 02:13:13.859835 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:13.859848 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:13.859958 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:13.860205 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:13.860207 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:13.860393 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:13.860494 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:13.860499 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:13.861222 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:13.861224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:13.861794 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:13.861802 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:13.862186 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:13.862188 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:13.862246 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:13.862725 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:13.862736 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:13.864612 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":160,"threads_started":1}
W20260806 02:13:13.864873 28273 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:13.859612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:56381 (local address 127.18.70.126:44741)
0806 02:13:13.859778 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:13.859786 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:13.859932 (+ 146us) server_negotiation.cc:407] Connection header received
0806 02:13:13.860044 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:13.860046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:13.860093 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:13.860161 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:13.860596 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:13.861114 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:13.861894 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:13.862094 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:13.862163 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:13.864742 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:56381: BlockingRecv error: recv got EOF from 127.18.70.66:56381 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 02:13:14 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:14 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:14.070353 28275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:14.064872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:50543)
0806 02:13:14.065217 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:14.065231 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:14.065339 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:14.065619 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:14.065624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:14.065826 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:14.065968 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:14.065975 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:14.066744 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:14.066748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:14.067318 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:14.067327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:14.067730 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:14.067733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:14.067801 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:14.068301 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:14.068312 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:14.070239 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":126,"threads_started":1}
W20260806 02:13:14.070518 28273 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:14.064949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:50543 (local address 127.18.70.126:44741)
0806 02:13:14.065159 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:14.065164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:14.065313 (+ 149us) server_negotiation.cc:407] Connection header received
0806 02:13:14.065418 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:14.065420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:14.065467 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:14.065534 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:14.066086 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:14.066629 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:14.067419 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:14.067639 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:14.067711 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:14.070366 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:50543: BlockingRecv error: recv got EOF from 127.18.70.67:50543 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 02:13:14 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:14 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:14.656121 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:14.650919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:41509)
0806 02:13:14.651206 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:14.651219 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:14.651334 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:14.651612 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:14.651615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:14.651802 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:14.651897 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:14.651902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:14.652600 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:14.652602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:14.653136 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:14.653143 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:14.653543 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:14.653546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:14.653605 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:14.654209 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:14.654221 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:14.656000 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":95,"threads_started":1}
W20260806 02:13:14.656329 28277 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:14.650999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:41509 (local address 127.18.70.126:44741)
0806 02:13:14.651273 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:14.651277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:14.651289 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:13:14.651429 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:14.651431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:14.651474 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:14.651617 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:14.652000 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:14.652505 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:14.653234 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:14.653442 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:14.653513 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:14.656139 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:41509: BlockingRecv error: recv got EOF from 127.18.70.65:41509 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":66,"threads_started":1}
Aug 06 02:13:14 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:14 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:14.871539 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:14.865841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:45003)
0806 02:13:14.866111 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:14.866127 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:14.866267 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:14.866570 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:14.866573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:14.866791 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:14.866886 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:14.866892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:14.867692 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:14.867697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:14.868246 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:14.868253 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:14.868723 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:14.868726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:14.868827 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:14.869455 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:14.869468 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:14.871397 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":84,"threads_started":1}
W20260806 02:13:14.871899 28277 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:14.865901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:45003 (local address 127.18.70.126:44741)
0806 02:13:14.866177 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:14.866202 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:14.866218 (+ 16us) server_negotiation.cc:407] Connection header received
0806 02:13:14.866389 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:14.866394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:14.866444 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:14.866542 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:14.867008 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:14.867557 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:14.868377 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:14.868606 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:14.868708 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:14.871633 (+ 2925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:45003: BlockingRecv error: recv got EOF from 127.18.70.66:45003 (error 108)
Metrics: {"server-negotiator.queue_time_us":175}
Aug 06 02:13:15 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:15 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:15.076678 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:15.071191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:49693)
0806 02:13:15.071531 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:15.071551 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:15.071694 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:15.071996 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:15.072000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:15.072204 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:15.072295 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:15.072300 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:15.073058 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:15.073063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:15.073630 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:15.073637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:15.074132 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:15.074136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:15.074223 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:15.074693 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:15.074705 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:15.076566 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":113,"threads_started":1}
W20260806 02:13:15.076822 28277 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:15.071409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:49693 (local address 127.18.70.126:44741)
0806 02:13:15.071530 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:15.071536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:15.071680 (+ 144us) server_negotiation.cc:407] Connection header received
0806 02:13:15.071827 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:15.071829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:15.071874 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:15.071967 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:15.072414 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:15.072943 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:15.073772 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:15.074018 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:15.074128 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:15.076695 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:49693: BlockingRecv error: recv got EOF from 127.18.70.67:49693 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 02:13:15 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:15 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:15.662220 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:15.656964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:34189)
0806 02:13:15.657236 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:15.657249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:15.657352 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:15.657599 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:15.657601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:15.657784 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:15.657882 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:15.657887 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:15.658645 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:15.658647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:15.659193 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:15.659200 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:15.659672 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:15.659675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:15.659735 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:15.660227 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:15.660237 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:15.662077 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":86,"threads_started":1}
W20260806 02:13:15.662344 28282 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:15.657029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:34189 (local address 127.18.70.126:44741)
0806 02:13:15.657276 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:15.657280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:15.657309 (+ 29us) server_negotiation.cc:407] Connection header received
0806 02:13:15.657439 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:15.657441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:15.657484 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:15.657563 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:15.657984 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:15.658544 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:15.659305 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:15.659529 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:15.659624 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:15.662205 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:34189: BlockingRecv error: recv got EOF from 127.18.70.65:34189 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":65,"threads_started":1}
Aug 06 02:13:15 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:15 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:15.877939 28283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:15.872453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:51745)
0806 02:13:15.872724 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:15.872756 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:15.872866 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:15.873152 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:15.873156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:15.873393 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:15.873517 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:15.873528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:15.874328 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:15.874333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:15.874927 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:15.874934 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:15.875408 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:15.875412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:15.875512 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:15.876025 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:15.876036 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:15.877827 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":94,"threads_started":1}
W20260806 02:13:15.878100 28282 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:15.872680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:51745 (local address 127.18.70.126:44741)
0806 02:13:15.872833 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:15.872837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:15.872851 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:13:15.872984 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:15.872989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:15.873039 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:15.873124 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:15.873651 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:15.874213 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:15.875048 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:15.875275 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:15.875403 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:15.877952 (+ 2549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:51745: BlockingRecv error: recv got EOF from 127.18.70.66:51745 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 02:13:16 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:16 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 02:13:16.083297 28282 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:16.077532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:56975 (local address 127.18.70.126:44741)
0806 02:13:16.077667 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:16.077671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:16.077887 (+ 216us) server_negotiation.cc:407] Connection header received
0806 02:13:16.078050 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:16.078053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:16.078110 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:16.078195 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:16.078655 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:16.079194 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:16.080072 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:16.080308 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:16.080418 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:16.083201 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:56975: BlockingRecv error: recv got EOF from 127.18.70.67:56975 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 02:13:16.083317 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:16.077486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:56975)
0806 02:13:16.077800 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:16.077812 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:16.077951 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:16.078217 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:16.078220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:16.078443 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:16.078537 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:16.078543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:16.079313 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:16.079319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:16.079955 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:16.079964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:16.080421 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:16.080425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:16.080509 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:16.081098 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:16.081110 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:16.083079 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":80,"threads_started":1}
Aug 06 02:13:16 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:16 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:16.668100 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:16.663052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:38429)
0806 02:13:16.663364 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:16.663377 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:16.663471 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:16.663752 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:16.663755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:16.663934 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:16.664035 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:16.664041 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:16.664732 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:16.664734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:16.665283 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:16.665291 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:16.665682 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:16.665685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:16.665753 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:16.666231 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:16.666242 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:16.667984 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":119,"threads_started":1}
W20260806 02:13:16.668256 28286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:16.663134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:38429 (local address 127.18.70.126:44741)
0806 02:13:16.663390 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:16.663394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:16.663426 (+ 32us) server_negotiation.cc:407] Connection header received
0806 02:13:16.663536 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:16.663538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:16.663611 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:16.663710 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:16.664139 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:16.664645 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:16.665380 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:16.665588 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:16.665661 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:16.668119 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:38429: BlockingRecv error: recv got EOF from 127.18.70.65:38429 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":99,"threads_started":1}
Aug 06 02:13:16 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:16 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:16.884204 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:16.878840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:50585)
0806 02:13:16.879145 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:16.879159 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:16.879274 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:16.879482 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:16.879484 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:16.879690 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:16.879788 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:16.879794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:16.880560 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:16.880564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:16.881270 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:16.881282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:16.881751 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:16.881755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:16.881836 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:16.882311 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:16.882323 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:16.884091 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":106,"threads_started":1}
W20260806 02:13:16.884339 28286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:16.878939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:50585 (local address 127.18.70.126:44741)
0806 02:13:16.879099 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:16.879104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:16.879231 (+ 127us) server_negotiation.cc:407] Connection header received
0806 02:13:16.879356 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:16.879358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:16.879402 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:16.879470 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:16.879883 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:16.880419 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:16.881398 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:16.881635 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:16.881755 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:16.884221 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:50585: BlockingRecv error: recv got EOF from 127.18.70.66:50585 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 02:13:17 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:17 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:17.094338 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:17.084153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:49661)
0806 02:13:17.084445 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:17.084459 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:17.084546 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:17.084819 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:17.084822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:17.084999 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:17.085098 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:17.085103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:17.085813 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:17.085816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:17.086392 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:17.086399 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:17.086818 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:17.086826 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:17.086900 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:17.092283 (+ 5383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:17.092294 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:17.094210 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":110,"threads_started":1}
W20260806 02:13:17.094499 28286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:17.084354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:49661 (local address 127.18.70.126:44741)
0806 02:13:17.084484 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:17.084487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:17.084503 (+ 16us) server_negotiation.cc:407] Connection header received
0806 02:13:17.084629 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:17.084631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:17.084675 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:17.084752 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:17.085194 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:17.085710 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:17.086489 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:17.086696 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:17.086771 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:17.094348 (+ 7577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:49661: BlockingRecv error: recv got EOF from 127.18.70.67:49661 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 02:13:17 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:17 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:17.674542 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:17.668920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:41749)
0806 02:13:17.669182 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:17.669195 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:17.669277 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:17.669520 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:17.669522 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:17.669696 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:17.669821 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:17.669826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:17.670533 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:17.670535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:17.671062 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:17.671069 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:17.671710 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:17.671717 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:17.671811 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:17.672402 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:17.672418 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:17.674427 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":93,"threads_started":1}
W20260806 02:13:17.674726 28290 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:17.669030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:41749 (local address 127.18.70.126:44741)
0806 02:13:17.669316 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:17.669319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:17.669330 (+ 11us) server_negotiation.cc:407] Connection header received
0806 02:13:17.669363 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:17.669365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:17.669404 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:17.669479 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:17.669921 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:17.670441 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:17.671197 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:17.671525 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:17.671652 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:17.674560 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:41749: BlockingRecv error: recv got EOF from 127.18.70.65:41749 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":98,"threads_started":1}
Aug 06 02:13:17 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:17 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:17.890455 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:17.885098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:44111)
0806 02:13:17.885404 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:17.885418 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:17.885515 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:17.885734 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:17.885737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:17.885927 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:17.886023 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:17.886029 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:17.886766 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:17.886776 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:17.887313 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:17.887320 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:17.887750 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:17.887752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:17.887821 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:17.888376 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:17.888387 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:17.890286 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":122,"threads_started":1}
W20260806 02:13:17.890563 28290 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:17.885223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:44111 (local address 127.18.70.126:44741)
0806 02:13:17.885375 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:17.885379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:17.885491 (+ 112us) server_negotiation.cc:407] Connection header received
0806 02:13:17.885581 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:17.885583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:17.885625 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:17.885696 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:17.886135 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:17.886674 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:17.887414 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:17.887659 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:17.887731 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:17.890431 (+ 2700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:44111: BlockingRecv error: recv got EOF from 127.18.70.66:44111 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 02:13:18 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:18 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:18.100885 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:18.095075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:33091)
0806 02:13:18.095404 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:18.095421 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:18.095537 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:18.095885 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:18.095888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:18.096134 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:18.096254 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:18.096261 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:18.097102 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:18.097107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:18.097668 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:18.097677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:18.098136 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:18.098140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:18.098225 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:18.098791 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:18.098802 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:18.100762 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":111,"threads_started":1}
W20260806 02:13:18.101042 28290 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:18.095306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:33091 (local address 127.18.70.126:44741)
0806 02:13:18.095455 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:18.095461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:18.095476 (+ 15us) server_negotiation.cc:407] Connection header received
0806 02:13:18.095702 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:18.095704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:18.095753 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:18.095853 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:18.096407 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:18.096989 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:18.097792 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:18.098019 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:18.098130 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:18.100889 (+ 2759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:33091: BlockingRecv error: recv got EOF from 127.18.70.67:33091 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 02:13:18 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:18 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:18.680581 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:18.675376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:33897)
0806 02:13:18.675724 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:18.675737 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:18.675831 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:18.676078 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:18.676080 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:18.676275 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:18.676377 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:18.676382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:18.677101 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:18.677104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:18.677683 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:18.677691 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:18.678105 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:18.678107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:18.678166 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:18.678669 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:18.678680 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:18.680451 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":93,"threads_started":1}
W20260806 02:13:18.680739 28294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:18.675445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:33897 (local address 127.18.70.126:44741)
0806 02:13:18.675779 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:18.675783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:18.675796 (+ 13us) server_negotiation.cc:407] Connection header received
0806 02:13:18.675918 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:18.675920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:18.675965 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:18.676039 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:18.676475 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:18.677009 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:18.677788 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:18.678008 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:18.678085 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:18.680581 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:33897: BlockingRecv error: recv got EOF from 127.18.70.65:33897 (error 108)
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":82,"threads_started":1}
Aug 06 02:13:18 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:18 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:18.897514 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:18.891721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:51031)
0806 02:13:18.892053 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:18.892070 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:18.892196 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:18.892462 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:18.892465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:18.892709 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:18.892859 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:18.892870 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:18.893816 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:18.893819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:18.894491 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:18.894501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:18.894899 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:18.894902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:18.894969 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:18.895469 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:18.895482 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:18.897401 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":133,"threads_started":1}
W20260806 02:13:18.897651 28294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:18.891755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:51031 (local address 127.18.70.126:44741)
0806 02:13:18.891905 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:18.891908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:18.892154 (+ 246us) server_negotiation.cc:407] Connection header received
0806 02:13:18.892294 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:18.892296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:18.892340 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:18.892409 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:18.893005 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:18.893691 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:18.894599 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:18.894796 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:18.894871 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:18.897528 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:51031: BlockingRecv error: recv got EOF from 127.18.70.66:51031 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 02:13:19 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:19 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.67@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:19.107165 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:19.101730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.67:51891)
0806 02:13:19.101999 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:19.102013 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:19.102144 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:19.102397 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:19.102399 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:19.102577 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:19.102732 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:19.102738 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:19.103442 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:19.103444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:19.103997 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:19.104004 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:19.104408 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:19.104411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:19.104487 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:19.105045 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:19.105057 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:19.107053 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":100,"threads_started":1}
W20260806 02:13:19.107306 28294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:19.101777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:51891 (local address 127.18.70.126:44741)
0806 02:13:19.101906 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:19.101910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:19.102098 (+ 188us) server_negotiation.cc:407] Connection header received
0806 02:13:19.102219 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:19.102221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:19.102268 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:19.102335 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:19.102836 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:19.103347 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:19.104103 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:19.104314 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:19.104390 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:19.107180 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.67:51891: BlockingRecv error: recv got EOF from 127.18.70.67:51891 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 02:13:19 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:19 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.65@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:19.687476 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:19.681553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.65:53309)
0806 02:13:19.681881 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:19.681896 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:19.681995 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:19.682266 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:19.682268 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:19.682477 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:19.682596 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:19.682602 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:19.683327 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:19.683330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:19.684060 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:19.684072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:19.684572 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:19.684576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:19.684667 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:19.685265 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:19.685281 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:19.687306 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":105,"threads_started":1}
W20260806 02:13:19.687585 28298 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:19.681668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:53309 (local address 127.18.70.126:44741)
0806 02:13:19.681930 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:19.681933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:19.681950 (+ 17us) server_negotiation.cc:407] Connection header received
0806 02:13:19.682093 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:19.682095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:19.682140 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:19.682217 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:19.682705 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:19.683225 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:19.684173 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:19.684429 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:19.684539 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:19.687435 (+ 2896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.65:53309: BlockingRecv error: recv got EOF from 127.18.70.65:53309 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":63,"threads_started":1}
Aug 06 02:13:19 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 02:13:19 dist-test-slave-6k22 krb5kdc[27195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.70.66@KRBTEST.COM for oryx/127.18.70.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 02:13:19.904385 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:19.898469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:44741 (local address 127.18.70.66:50935)
0806 02:13:19.898822 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:19.898841 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:19.898965 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:19.899234 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:19.899237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:19.899445 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:19.899537 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:19.899544 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:19.900352 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:19.900357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:19.900929 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:19.900937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:19.901507 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:19.901510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:19.901615 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:19.902114 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:19.902125 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:19.904244 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.70.126:44741: Server oryx/127.18.70.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":107,"threads_started":1}
W20260806 02:13:19.904500 28298 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:19.898592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:50935 (local address 127.18.70.126:44741)
0806 02:13:19.898792 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:19.898797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:19.898913 (+ 116us) server_negotiation.cc:407] Connection header received
0806 02:13:19.899063 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:19.899067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:19.899115 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:19.899210 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:19.899703 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:19.900258 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:19.901050 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:19.901378 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:19.901493 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:19.904387 (+ 2894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.70.66:50935: BlockingRecv error: recv got EOF from 127.18.70.66:50935 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 02:13:19.995317 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 27802
I20260806 02:13:20.003870 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 27963
I20260806 02:13:20.012094 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 28100
I20260806 02:13:20.018798 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 27729
2026-08-06T02:13:20Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18048 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T02:13:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:13:20Z Disabled control of system clock
I20260806 02:13:20.052287 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:37419
--webserver_interface=127.18.70.126
--webserver_port=0
--builtin_ntp_servers=127.18.70.84:44527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:37419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 02:13:20.161998 28303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:20.162238 28303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:20.162272 28303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:20.166007 28303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:13:20.166051 28303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:20.166065 28303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:20.166079 28303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:13:20.166092 28303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:13:20.170742 28303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:44527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:37419
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:37419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:20.171766 28303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:20.173483 28303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:20.184353 28308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:20.184407 28309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:20.184798 28311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:20.185163 28303 server_base.cc:1034] running on GCE node
I20260806 02:13:20.185876 28303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:20.187414 28303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:20.188575 28303 hybrid_clock.cc:648] HybridClock initialized: now 1785982400188504 us; error 107 us; skew 500 ppm
I20260806 02:13:20.191000 28303 webserver.cc:460] Webserver started at http://127.18.70.126:43653/ using document root <none> and password file <none>
I20260806 02:13:20.191752 28303 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:20.191850 28303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:20.192085 28303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:20.194856 28303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "0b980bf7c3e749e89127aea70df4e703"
format_stamp: "Formatted at 2026-08-06 02:13:20 on dist-test-slave-6k22"
server_key: "3f34c1461153d5015d2a052e920da06c"
server_key_iv: "d26d6376c0a8c60cccb02d38262ae2ee"
server_key_version: "encryptionkey@0"
I20260806 02:13:20.195434 28303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "0b980bf7c3e749e89127aea70df4e703"
format_stamp: "Formatted at 2026-08-06 02:13:20 on dist-test-slave-6k22"
server_key: "3f34c1461153d5015d2a052e920da06c"
server_key_iv: "d26d6376c0a8c60cccb02d38262ae2ee"
server_key_version: "encryptionkey@0"
I20260806 02:13:20.200537 28303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 02:13:20.203263 28317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:20.204558 28303 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 02:13:20.204691 28303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "0b980bf7c3e749e89127aea70df4e703"
format_stamp: "Formatted at 2026-08-06 02:13:20 on dist-test-slave-6k22"
server_key: "3f34c1461153d5015d2a052e920da06c"
server_key_iv: "d26d6376c0a8c60cccb02d38262ae2ee"
server_key_version: "encryptionkey@0"
I20260806 02:13:20.204813 28303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:20.249308 28303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:20.250084 28303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:20.250298 28303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:20.257823 28303 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:37419
I20260806 02:13:20.257849 28371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:37419 every 8 connection(s)
I20260806 02:13:20.258890 28303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:13:20.262405 28372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:13:20.267942 28372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703: Bootstrap starting.
I20260806 02:13:20.268415 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 28303
I20260806 02:13:20.268509 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:13:20.268782 18713 external_mini_cluster.cc:1428] Setting key 151eeb6c3b79ff2b77002f04b8278a46
I20260806 02:13:20.270764 28372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703: Neither blocks nor log segments found. Creating new log.
I20260806 02:13:20.271549 28372 log.cc:824] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703: Log is configured to *not* fsync() on all Append() calls
I20260806 02:13:20.273813 28372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703: No bootstrap required, opened a new log
I20260806 02:13:20.278234 28372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b980bf7c3e749e89127aea70df4e703" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37419 } }
I20260806 02:13:20.278514 28372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:13:20.278589 28372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b980bf7c3e749e89127aea70df4e703, State: Initialized, Role: FOLLOWER
I20260806 02:13:20.279066 28372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b980bf7c3e749e89127aea70df4e703" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37419 } }
I20260806 02:13:20.279172 28375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:20.270221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38162 (local address 127.18.70.126:37419)
0806 02:13:20.270890 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:20.270911 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:20.270958 (+ 47us) server_negotiation.cc:407] Connection header received
0806 02:13:20.271641 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:20.271668 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:20.272104 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:20.272420 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:20.272846 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.274242 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.275180 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.275933 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.276127 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:20.276947 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:20.276955 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:20.276977 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:13:20.277563 (+ 586us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:20.277633 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:13:20.277660 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:20.277762 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:20.277915 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 02:13:20.278183 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":150,"threads_started":1}
I20260806 02:13:20.279227 28372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:13:20.279284 28372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:13:20.279385 28372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:20.280550 28372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b980bf7c3e749e89127aea70df4e703" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37419 } }
I20260806 02:13:20.280892 28372 leader_election.cc:302] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b980bf7c3e749e89127aea70df4e703; no voters:
I20260806 02:13:20.281188 28372 leader_election.cc:288] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:13:20.281296 28378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:13:20.281530 28378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [term 1 LEADER]: Becoming Leader. State: Replica: 0b980bf7c3e749e89127aea70df4e703, State: Running, Role: LEADER
I20260806 02:13:20.281904 28378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b980bf7c3e749e89127aea70df4e703" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37419 } }
I20260806 02:13:20.282425 28372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:13:20.283630 28380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b980bf7c3e749e89127aea70df4e703. Latest consensus state: current_term: 1 leader_uuid: "0b980bf7c3e749e89127aea70df4e703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b980bf7c3e749e89127aea70df4e703" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37419 } } }
I20260806 02:13:20.283756 28380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [sys.catalog]: This master's current role is: LEADER
I20260806 02:13:20.284051 28379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b980bf7c3e749e89127aea70df4e703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b980bf7c3e749e89127aea70df4e703" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37419 } } }
I20260806 02:13:20.284153 28379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703 [sys.catalog]: This master's current role is: LEADER
I20260806 02:13:20.284237 28387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:13:20.287365 28387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:13:20.292574 28387 catalog_manager.cc:1347] Generated new cluster ID: 04875a3a797a4b918f96a38b181cf7db
I20260806 02:13:20.292636 28387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:13:20.318991 28387 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:13:20.320003 28387 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:13:20.325973 28387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b980bf7c3e749e89127aea70df4e703: Generated new TSK 0
I20260806 02:13:20.326653 28387 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:13:20.331482 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37419
--builtin_ntp_servers=127.18.70.84:44527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 02:13:20.442770 28398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:20.443048 28398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:20.443116 28398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:20.446954 28398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:20.447033 28398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:20.447124 28398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:13:20.451462 28398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:44527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:20.452481 28398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:20.453729 28398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:20.463131 28403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:20.463145 28406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:20.463263 28398 server_base.cc:1034] running on GCE node
W20260806 02:13:20.463430 28404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:20.463840 28398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:20.464486 28398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:20.465672 28398 hybrid_clock.cc:648] HybridClock initialized: now 1785982400465644 us; error 42 us; skew 500 ppm
I20260806 02:13:20.467463 28398 webserver.cc:460] Webserver started at http://127.18.70.65:37781/ using document root <none> and password file <none>
I20260806 02:13:20.468055 28398 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:20.468153 28398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:20.468390 28398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:20.470459 28398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "1c40147112294b879bd8c029c5481382"
format_stamp: "Formatted at 2026-08-06 02:13:20 on dist-test-slave-6k22"
server_key: "c75777ca1417cc6427247e8c0c1cf88d"
server_key_iv: "3a2b5867959c00a9b68f5aba9dd61814"
server_key_version: "encryptionkey@0"
I20260806 02:13:20.471031 28398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "1c40147112294b879bd8c029c5481382"
format_stamp: "Formatted at 2026-08-06 02:13:20 on dist-test-slave-6k22"
server_key: "c75777ca1417cc6427247e8c0c1cf88d"
server_key_iv: "3a2b5867959c00a9b68f5aba9dd61814"
server_key_version: "encryptionkey@0"
I20260806 02:13:20.474738 28398 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 02:13:20.477227 28413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:20.478434 28398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:13:20.478565 28398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "1c40147112294b879bd8c029c5481382"
format_stamp: "Formatted at 2026-08-06 02:13:20 on dist-test-slave-6k22"
server_key: "c75777ca1417cc6427247e8c0c1cf88d"
server_key_iv: "3a2b5867959c00a9b68f5aba9dd61814"
server_key_version: "encryptionkey@0"
I20260806 02:13:20.478677 28398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:20.514675 28398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:20.515403 28398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:20.515643 28398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:20.516237 28398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:20.517205 28398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:20.517279 28398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:20.517347 28398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:20.517400 28398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:20.526459 28398 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:45777
I20260806 02:13:20.526532 28540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:45777 every 8 connection(s)
I20260806 02:13:20.527383 28398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:13:20.528883 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 28398
I20260806 02:13:20.529014 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:13:20.529336 18713 external_mini_cluster.cc:1428] Setting key ed7d5de03e3de64e0d0e54a62636d2a7
I20260806 02:13:20.531435 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37419
--builtin_ntp_servers=127.18.70.84:44527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 02:13:20.538542 28375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:20.529204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:40605 (local address 127.18.70.126:37419)
0806 02:13:20.529362 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:20.529367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:20.530536 (+ 1169us) server_negotiation.cc:407] Connection header received
0806 02:13:20.532487 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:20.532490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:20.532530 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:20.532595 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:20.533673 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.534564 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.536332 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.536613 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.536708 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:20.538056 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:20.538060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:20.538062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:13:20.538074 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:20.538093 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:13:20.538101 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:20.538160 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:20.538384 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 02:13:20.538436 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 02:13:20.539842 28543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:20.529649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:37419 (local address 127.18.70.65:40605)
0806 02:13:20.530398 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:20.530437 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:20.532224 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:20.532762 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:20.532772 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:20.532874 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:20.533527 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:20.533542 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.534703 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.534706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:20.536191 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:20.536207 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.536791 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.536798 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:20.537069 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:20.537729 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:13:20.537733 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:20.537795 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:13:20.537887 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:13:20.538192 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:13:20.538205 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:13:20.538214 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 02:13:20.538385 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 02:13:20.538710 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":135,"threads_started":1}
I20260806 02:13:20.541126 28541 heartbeater.cc:347] Connected to a master server at 127.18.70.126:37419
I20260806 02:13:20.541356 28541 heartbeater.cc:464] Registering TS with master...
I20260806 02:13:20.541858 28541 heartbeater.cc:511] Master 127.18.70.126:37419 requested a full tablet report, sending...
I20260806 02:13:20.543363 28336 ts_manager.cc:194] Registered new tserver with Master: 1c40147112294b879bd8c029c5481382 (127.18.70.65:45777)
I20260806 02:13:20.544476 28336 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.70.65:40605
W20260806 02:13:20.645892 28544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:20.646164 28544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:20.646229 28544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:20.650095 28544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:20.650166 28544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:20.650249 28544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:13:20.654747 28544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:44527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:20.655813 28544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:20.657150 28544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 02:13:20.667032 28544 server_base.cc:1034] running on GCE node
W20260806 02:13:20.666985 28550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:20.666961 28549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:20.666980 28553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:20.667618 28544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:20.668284 28544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:20.669474 28544 hybrid_clock.cc:648] HybridClock initialized: now 1785982400669455 us; error 31 us; skew 500 ppm
I20260806 02:13:20.671193 28544 webserver.cc:460] Webserver started at http://127.18.70.66:46465/ using document root <none> and password file <none>
I20260806 02:13:20.671813 28544 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:20.671900 28544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:20.672137 28544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:20.674175 28544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "554ec0e782a841ee96b6ee868366c898"
format_stamp: "Formatted at 2026-08-06 02:13:20 on dist-test-slave-6k22"
server_key: "2294552bfeb2621c79a14796c22a5df6"
server_key_iv: "1f22e3524cc9b49824816ce3a67865d3"
server_key_version: "encryptionkey@0"
I20260806 02:13:20.674702 28544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "554ec0e782a841ee96b6ee868366c898"
format_stamp: "Formatted at 2026-08-06 02:13:20 on dist-test-slave-6k22"
server_key: "2294552bfeb2621c79a14796c22a5df6"
server_key_iv: "1f22e3524cc9b49824816ce3a67865d3"
server_key_version: "encryptionkey@0"
I20260806 02:13:20.678248 28544 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 02:13:20.680686 28559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:20.681882 28544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:13:20.681998 28544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "554ec0e782a841ee96b6ee868366c898"
format_stamp: "Formatted at 2026-08-06 02:13:20 on dist-test-slave-6k22"
server_key: "2294552bfeb2621c79a14796c22a5df6"
server_key_iv: "1f22e3524cc9b49824816ce3a67865d3"
server_key_version: "encryptionkey@0"
I20260806 02:13:20.682101 28544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:20.694247 28544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:20.694885 28544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:20.695077 28544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:20.695673 28544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:20.696594 28544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:20.696663 28544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:20.696732 28544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:20.696787 28544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:20.705593 28544 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:42309
I20260806 02:13:20.705644 28685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:42309 every 8 connection(s)
I20260806 02:13:20.706492 28544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:13:20.709298 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 28544
I20260806 02:13:20.709427 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:13:20.709735 18713 external_mini_cluster.cc:1428] Setting key 08be7f01d4984836538b6dbce80077dc
I20260806 02:13:20.711997 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37419
--builtin_ntp_servers=127.18.70.84:44527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 02:13:20.716663 28375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:20.708327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:40351 (local address 127.18.70.126:37419)
0806 02:13:20.708450 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:20.708454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:20.709775 (+ 1321us) server_negotiation.cc:407] Connection header received
0806 02:13:20.711427 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:20.711429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:20.711471 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:20.711532 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:20.712737 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.713238 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.714506 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.714879 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.714996 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:20.716056 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:20.716063 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:20.716069 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:13:20.716089 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:20.716115 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:13:20.716131 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:20.716222 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:20.716384 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 02:13:20.716449 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 02:13:20.717514 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:20.708733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:37419 (local address 127.18.70.66:40351)
0806 02:13:20.709605 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:20.709659 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:20.711197 (+ 1538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:20.711762 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:20.711772 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:20.711880 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:20.712593 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:20.712614 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.713366 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.713373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:20.714380 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:20.714390 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.715026 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.715029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:20.715160 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:20.715797 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:13:20.715799 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:20.715835 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:13:20.715892 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:13:20.716224 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:13:20.716232 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:13:20.716236 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 02:13:20.716350 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 02:13:20.716620 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":115,"threads_started":1}
I20260806 02:13:20.718683 28686 heartbeater.cc:347] Connected to a master server at 127.18.70.126:37419
I20260806 02:13:20.718915 28686 heartbeater.cc:464] Registering TS with master...
I20260806 02:13:20.719422 28686 heartbeater.cc:511] Master 127.18.70.126:37419 requested a full tablet report, sending...
I20260806 02:13:20.720487 28336 ts_manager.cc:194] Registered new tserver with Master: 554ec0e782a841ee96b6ee868366c898 (127.18.70.66:42309)
I20260806 02:13:20.721160 28336 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.70.66:40351
W20260806 02:13:20.826712 28689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:20.826972 28689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:20.827050 28689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:20.830963 28689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:20.831014 28689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:20.831065 28689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:13:20.835693 28689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:44527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:20.836715 28689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:20.838150 28689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:20.848052 28694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:20.848181 28695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:20.848169 28697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:20.848223 28689 server_base.cc:1034] running on GCE node
I20260806 02:13:20.848752 28689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:20.849385 28689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:20.850539 28689 hybrid_clock.cc:648] HybridClock initialized: now 1785982400850504 us; error 46 us; skew 500 ppm
I20260806 02:13:20.852309 28689 webserver.cc:460] Webserver started at http://127.18.70.67:33363/ using document root <none> and password file <none>
I20260806 02:13:20.852891 28689 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:20.852952 28689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:20.853206 28689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:20.855199 28689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "9b7d1d8df7ce44979428c10707df64fb"
format_stamp: "Formatted at 2026-08-06 02:13:20 on dist-test-slave-6k22"
server_key: "c95bdacd10c017efca0cba96c1b2e672"
server_key_iv: "21e3ff580a25ab9f0abe51c2b13fe1d8"
server_key_version: "encryptionkey@0"
I20260806 02:13:20.855783 28689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "9b7d1d8df7ce44979428c10707df64fb"
format_stamp: "Formatted at 2026-08-06 02:13:20 on dist-test-slave-6k22"
server_key: "c95bdacd10c017efca0cba96c1b2e672"
server_key_iv: "21e3ff580a25ab9f0abe51c2b13fe1d8"
server_key_version: "encryptionkey@0"
I20260806 02:13:20.859341 28689 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 02:13:20.861819 28704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:20.862969 28689 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:13:20.863080 28689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "9b7d1d8df7ce44979428c10707df64fb"
format_stamp: "Formatted at 2026-08-06 02:13:20 on dist-test-slave-6k22"
server_key: "c95bdacd10c017efca0cba96c1b2e672"
server_key_iv: "21e3ff580a25ab9f0abe51c2b13fe1d8"
server_key_version: "encryptionkey@0"
I20260806 02:13:20.863183 28689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:20.891368 28689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:20.892108 28689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:20.892256 28689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:20.892802 28689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:20.893687 28689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:20.893734 28689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:20.893774 28689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:20.893791 28689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:20.902340 28689 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:37155
I20260806 02:13:20.902367 28821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:37155 every 8 connection(s)
I20260806 02:13:20.903231 28689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:13:20.909121 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 28689
I20260806 02:13:20.909219 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:13:20.909475 18713 external_mini_cluster.cc:1428] Setting key e371f0e73aea3dc5e02690bceb98cc58
I20260806 02:13:20.912751 28375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:20.905084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:51595 (local address 127.18.70.126:37419)
0806 02:13:20.905222 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:20.905225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:20.906222 (+ 997us) server_negotiation.cc:407] Connection header received
0806 02:13:20.907697 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:20.907700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:20.907793 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:20.907896 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:20.908760 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.909462 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.910888 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.911101 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.911208 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:20.912250 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:20.912254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:20.912256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:13:20.912270 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:20.912295 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:13:20.912303 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:20.912397 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:20.912558 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 02:13:20.912617 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 02:13:20.913578 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:20.905452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:37419 (local address 127.18.70.67:51595)
0806 02:13:20.906078 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:20.906115 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:20.907420 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:20.908083 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:20.908090 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:20.908169 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:20.908620 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:20.908632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.909620 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.909623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:20.910760 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:20.910770 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:20.911246 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:20.911250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:20.911423 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:20.912006 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:13:20.912008 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:20.912046 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:13:20.912103 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:13:20.912412 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:13:20.912429 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:13:20.912435 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 02:13:20.912556 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 02:13:20.912823 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":112,"threads_started":1}
I20260806 02:13:20.914603 28823 heartbeater.cc:347] Connected to a master server at 127.18.70.126:37419
I20260806 02:13:20.914842 28823 heartbeater.cc:464] Registering TS with master...
I20260806 02:13:20.915362 28823 heartbeater.cc:511] Master 127.18.70.126:37419 requested a full tablet report, sending...
I20260806 02:13:20.916380 28336 ts_manager.cc:194] Registered new tserver with Master: 9b7d1d8df7ce44979428c10707df64fb (127.18.70.67:37155)
I20260806 02:13:20.916914 28336 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.70.67:51595
I20260806 02:13:20.923170 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 02:13:20.925908 28375 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:20.925113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38178 (local address 127.18.70.126:37419)
0806 02:13:20.925214 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:20.925217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:20.925407 (+ 190us) server_negotiation.cc:407] Connection header received
0806 02:13:20.925485 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:20.925488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:20.925520 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:20.925580 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:20.925804 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38178: BlockingRecv error: recv got EOF from 127.0.0.1:38178 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 02:13:20.926371 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 28398
I20260806 02:13:20.936659 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 28544
I20260806 02:13:20.944262 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 28689
I20260806 02:13:20.950304 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 28303
2026-08-06T02:13:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (929 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T02:13:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:13:20Z Disabled control of system clock
I20260806 02:13:20.980947 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:33227
--webserver_interface=127.18.70.126
--webserver_port=0
--builtin_ntp_servers=127.18.70.84:41607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:33227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 02:13:21.099547 28834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:21.099858 28834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:21.099895 28834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:21.103871 28834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:13:21.103926 28834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:21.103941 28834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:21.103958 28834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:13:21.103973 28834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:13:21.108677 28834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:33227
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:33227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:21.109696 28834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:21.111102 28834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:21.119861 28843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:21.120039 28840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:21.119860 28841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:21.120121 28834 server_base.cc:1034] running on GCE node
I20260806 02:13:21.120800 28834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:21.121886 28834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:21.123059 28834 hybrid_clock.cc:648] HybridClock initialized: now 1785982401123033 us; error 45 us; skew 500 ppm
I20260806 02:13:21.125049 28834 webserver.cc:460] Webserver started at http://127.18.70.126:38757/ using document root <none> and password file <none>
I20260806 02:13:21.125622 28834 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:21.125717 28834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:21.125957 28834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:21.128087 28834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "8810ef25393c444a9d2cb5e73e0eeaf1"
format_stamp: "Formatted at 2026-08-06 02:13:21 on dist-test-slave-6k22"
server_key: "076208aed4373feec639aa0b6ac2b5ac"
server_key_iv: "4714a7a289bdb9cb45fa8d422d34f7c4"
server_key_version: "encryptionkey@0"
I20260806 02:13:21.128656 28834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "8810ef25393c444a9d2cb5e73e0eeaf1"
format_stamp: "Formatted at 2026-08-06 02:13:21 on dist-test-slave-6k22"
server_key: "076208aed4373feec639aa0b6ac2b5ac"
server_key_iv: "4714a7a289bdb9cb45fa8d422d34f7c4"
server_key_version: "encryptionkey@0"
I20260806 02:13:21.132318 28834 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:13:21.134614 28849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:21.135870 28834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:13:21.136003 28834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "8810ef25393c444a9d2cb5e73e0eeaf1"
format_stamp: "Formatted at 2026-08-06 02:13:21 on dist-test-slave-6k22"
server_key: "076208aed4373feec639aa0b6ac2b5ac"
server_key_iv: "4714a7a289bdb9cb45fa8d422d34f7c4"
server_key_version: "encryptionkey@0"
I20260806 02:13:21.136108 28834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:21.140552 28834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:21.141146 28834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:21.141326 28834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:21.148397 28834 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:33227
I20260806 02:13:21.148418 28903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:33227 every 8 connection(s)
I20260806 02:13:21.149245 28834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:13:21.151913 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 28834
I20260806 02:13:21.152048 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:13:21.152323 18713 external_mini_cluster.cc:1428] Setting key 2d482284fe1d15c4ec13802140e89f86
I20260806 02:13:21.152356 28904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:13:21.157608 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:21.153680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36982 (local address 127.18.70.126:33227)
0806 02:13:21.154284 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:21.154298 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:21.154332 (+ 34us) server_negotiation.cc:407] Connection header received
0806 02:13:21.154938 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:21.154953 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:21.155278 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:21.155549 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:21.156386 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:21.156391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:21.156402 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:13:21.156412 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:21.156462 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:13:21.156477 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:21.156590 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:21.156721 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 02:13:21.156985 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":132,"threads_started":1}
I20260806 02:13:21.159035 28904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1: Bootstrap starting.
I20260806 02:13:21.161235 28904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1: Neither blocks nor log segments found. Creating new log.
I20260806 02:13:21.161945 28904 log.cc:824] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1: Log is configured to *not* fsync() on all Append() calls
I20260806 02:13:21.163668 28904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1: No bootstrap required, opened a new log
I20260806 02:13:21.166894 28904 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8810ef25393c444a9d2cb5e73e0eeaf1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33227 } }
I20260806 02:13:21.167150 28904 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:13:21.167215 28904 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8810ef25393c444a9d2cb5e73e0eeaf1, State: Initialized, Role: FOLLOWER
I20260806 02:13:21.167668 28904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8810ef25393c444a9d2cb5e73e0eeaf1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33227 } }
I20260806 02:13:21.167824 28904 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:13:21.167866 28904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:13:21.167939 28904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:21.168606 28904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8810ef25393c444a9d2cb5e73e0eeaf1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33227 } }
I20260806 02:13:21.168900 28904 leader_election.cc:302] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8810ef25393c444a9d2cb5e73e0eeaf1; no voters:
I20260806 02:13:21.169116 28904 leader_election.cc:288] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:13:21.169246 28909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:13:21.169520 28909 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [term 1 LEADER]: Becoming Leader. State: Replica: 8810ef25393c444a9d2cb5e73e0eeaf1, State: Running, Role: LEADER
I20260806 02:13:21.169853 28909 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8810ef25393c444a9d2cb5e73e0eeaf1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33227 } }
I20260806 02:13:21.170316 28904 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:13:21.171586 28911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8810ef25393c444a9d2cb5e73e0eeaf1. Latest consensus state: current_term: 1 leader_uuid: "8810ef25393c444a9d2cb5e73e0eeaf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8810ef25393c444a9d2cb5e73e0eeaf1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33227 } } }
I20260806 02:13:21.171732 28911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [sys.catalog]: This master's current role is: LEADER
I20260806 02:13:21.171593 28910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8810ef25393c444a9d2cb5e73e0eeaf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8810ef25393c444a9d2cb5e73e0eeaf1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33227 } } }
I20260806 02:13:21.172086 28910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1 [sys.catalog]: This master's current role is: LEADER
I20260806 02:13:21.172175 28919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:13:21.175103 28919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:13:21.180543 28919 catalog_manager.cc:1347] Generated new cluster ID: 753e9d97f2524aaebe42adf47e133fae
I20260806 02:13:21.180603 28919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:13:21.194056 28919 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:13:21.194370 28919 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:13:21.206686 28919 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8810ef25393c444a9d2cb5e73e0eeaf1: Generated new TSK 0
I20260806 02:13:21.207295 28919 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:13:21.211170 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33227
--builtin_ntp_servers=127.18.70.84:41607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 02:13:21.323113 28930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:21.323331 28930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:21.323366 28930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:21.327350 28930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:21.327397 28930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:21.327500 28930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:13:21.332125 28930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:21.333098 28930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:21.334419 28930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:21.343971 28935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:21.344009 28930 server_base.cc:1034] running on GCE node
W20260806 02:13:21.343977 28936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:21.343983 28938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:21.344515 28930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:21.345118 28930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:21.346305 28930 hybrid_clock.cc:648] HybridClock initialized: now 1785982401346291 us; error 24 us; skew 500 ppm
I20260806 02:13:21.348265 28930 webserver.cc:460] Webserver started at http://127.18.70.65:36689/ using document root <none> and password file <none>
I20260806 02:13:21.348830 28930 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:21.348922 28930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:21.349150 28930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:21.351153 28930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "fc7f38d6ecbc4b159ba1055161a5fe73"
format_stamp: "Formatted at 2026-08-06 02:13:21 on dist-test-slave-6k22"
server_key: "f37dc6d017b67595ec08723364e9c664"
server_key_iv: "e7c747b9ba1e23c1be010d52718d3589"
server_key_version: "encryptionkey@0"
I20260806 02:13:21.351796 28930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "fc7f38d6ecbc4b159ba1055161a5fe73"
format_stamp: "Formatted at 2026-08-06 02:13:21 on dist-test-slave-6k22"
server_key: "f37dc6d017b67595ec08723364e9c664"
server_key_iv: "e7c747b9ba1e23c1be010d52718d3589"
server_key_version: "encryptionkey@0"
I20260806 02:13:21.355353 28930 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:13:21.357817 28944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:21.359006 28930 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:13:21.359140 28930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "fc7f38d6ecbc4b159ba1055161a5fe73"
format_stamp: "Formatted at 2026-08-06 02:13:21 on dist-test-slave-6k22"
server_key: "f37dc6d017b67595ec08723364e9c664"
server_key_iv: "e7c747b9ba1e23c1be010d52718d3589"
server_key_version: "encryptionkey@0"
I20260806 02:13:21.359251 28930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:21.363709 28930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:21.364285 28930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:21.364465 28930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:21.365008 28930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:21.365898 28930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:21.365969 28930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:21.366030 28930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:21.366082 28930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:21.375135 28930 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:38975
I20260806 02:13:21.375159 29065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:38975 every 8 connection(s)
I20260806 02:13:21.376073 28930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:13:21.378014 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 28930
I20260806 02:13:21.378132 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:13:21.378468 18713 external_mini_cluster.cc:1428] Setting key d957ecfa3d9c5fbfc62258194ec3ec4e
I20260806 02:13:21.380821 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33227
--builtin_ntp_servers=127.18.70.84:41607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 02:13:21.383018 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:21.377943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:51663 (local address 127.18.70.126:33227)
0806 02:13:21.379401 (+ 1458us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:21.379405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:21.379508 (+ 103us) server_negotiation.cc:407] Connection header received
0806 02:13:21.381150 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:21.381155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:21.381202 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:21.381291 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:21.382488 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:21.382492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:21.382494 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:13:21.382506 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:21.382528 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:13:21.382536 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:21.382589 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:21.382801 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 02:13:21.382854 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1377}
I20260806 02:13:21.383965 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:21.378457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:33227 (local address 127.18.70.65:51663)
0806 02:13:21.379356 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:21.379408 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:21.380903 (+ 1495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:21.381470 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:21.381478 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:21.381580 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:21.382200 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:13:21.382202 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:21.382251 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:13:21.382331 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:13:21.382644 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:13:21.382651 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:13:21.382656 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 02:13:21.382812 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 02:13:21.383039 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":122,"threads_started":1}
I20260806 02:13:21.386351 29066 heartbeater.cc:347] Connected to a master server at 127.18.70.126:33227
I20260806 02:13:21.386618 29066 heartbeater.cc:464] Registering TS with master...
I20260806 02:13:21.387133 29066 heartbeater.cc:511] Master 127.18.70.126:33227 requested a full tablet report, sending...
I20260806 02:13:21.388808 28866 ts_manager.cc:194] Registered new tserver with Master: fc7f38d6ecbc4b159ba1055161a5fe73 (127.18.70.65:38975)
W20260806 02:13:21.494362 29069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:21.494585 29069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:21.494619 29069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:21.498582 29069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:21.498629 29069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:21.498682 29069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:13:21.503137 29069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:21.504156 29069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:21.505510 29069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:21.514911 29077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:21.514963 29074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:21.515074 29069 server_base.cc:1034] running on GCE node
W20260806 02:13:21.515255 29075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:21.515723 29069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:21.516373 29069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:21.517544 29069 hybrid_clock.cc:648] HybridClock initialized: now 1785982401517515 us; error 42 us; skew 500 ppm
I20260806 02:13:21.519260 29069 webserver.cc:460] Webserver started at http://127.18.70.66:40207/ using document root <none> and password file <none>
I20260806 02:13:21.519852 29069 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:21.519914 29069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:21.520112 29069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:21.521993 29069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "a5601614bbcd41219894846f74dac728"
format_stamp: "Formatted at 2026-08-06 02:13:21 on dist-test-slave-6k22"
server_key: "77b1fded30dffa3d956aa2e7ccc884f7"
server_key_iv: "d870f4e24213b6a1f813b00d64f653b7"
server_key_version: "encryptionkey@0"
I20260806 02:13:21.522490 29069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "a5601614bbcd41219894846f74dac728"
format_stamp: "Formatted at 2026-08-06 02:13:21 on dist-test-slave-6k22"
server_key: "77b1fded30dffa3d956aa2e7ccc884f7"
server_key_iv: "d870f4e24213b6a1f813b00d64f653b7"
server_key_version: "encryptionkey@0"
I20260806 02:13:21.526165 29069 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 02:13:21.528788 29083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:21.529942 29069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:13:21.530071 29069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "a5601614bbcd41219894846f74dac728"
format_stamp: "Formatted at 2026-08-06 02:13:21 on dist-test-slave-6k22"
server_key: "77b1fded30dffa3d956aa2e7ccc884f7"
server_key_iv: "d870f4e24213b6a1f813b00d64f653b7"
server_key_version: "encryptionkey@0"
I20260806 02:13:21.530184 29069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:21.534768 29069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:21.535352 29069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:21.535529 29069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:21.536108 29069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:21.536988 29069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:21.537067 29069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:21.537154 29069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:21.537204 29069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:21.545782 29069 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:36933
I20260806 02:13:21.545861 29201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:36933 every 8 connection(s)
I20260806 02:13:21.546667 29069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:13:21.548594 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 29069
I20260806 02:13:21.548749 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:13:21.549037 18713 external_mini_cluster.cc:1428] Setting key 5d9bd7c71af5d017bf4088cde6e2aedd
I20260806 02:13:21.551219 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33227
--builtin_ntp_servers=127.18.70.84:41607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 02:13:21.553627 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:21.548616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:35167 (local address 127.18.70.126:33227)
0806 02:13:21.548802 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:21.548807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:21.549850 (+ 1043us) server_negotiation.cc:407] Connection header received
0806 02:13:21.551499 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:21.551504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:21.551551 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:21.551628 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:21.553080 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:21.553086 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:21.553091 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:13:21.553108 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:21.553150 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:13:21.553162 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:21.553277 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:21.553409 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 02:13:21.553458 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 02:13:21.554391 29204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:21.548950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:33227 (local address 127.18.70.66:35167)
0806 02:13:21.549681 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:21.549732 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:21.551209 (+ 1477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:21.552044 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:21.552055 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:21.552167 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:21.552736 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:13:21.552740 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:21.552803 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:13:21.552893 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:13:21.553269 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:13:21.553278 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:13:21.553283 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 02:13:21.553415 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 02:13:21.553597 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":133,"threads_started":1}
I20260806 02:13:21.555548 29202 heartbeater.cc:347] Connected to a master server at 127.18.70.126:33227
I20260806 02:13:21.555815 29202 heartbeater.cc:464] Registering TS with master...
I20260806 02:13:21.556275 29202 heartbeater.cc:511] Master 127.18.70.126:33227 requested a full tablet report, sending...
I20260806 02:13:21.557364 28866 ts_manager.cc:194] Registered new tserver with Master: a5601614bbcd41219894846f74dac728 (127.18.70.66:36933)
W20260806 02:13:21.662869 29205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:21.663220 29205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:21.663277 29205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:21.667238 29205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:21.667286 29205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:21.667339 29205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:13:21.672492 29205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:21.673846 29205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:21.675236 29205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:21.684518 29211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:21.684523 29210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:21.684651 29205 server_base.cc:1034] running on GCE node
W20260806 02:13:21.684516 29213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:21.685138 29205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:21.685757 29205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:21.686919 29205 hybrid_clock.cc:648] HybridClock initialized: now 1785982401686905 us; error 25 us; skew 500 ppm
I20260806 02:13:21.688879 29205 webserver.cc:460] Webserver started at http://127.18.70.67:41893/ using document root <none> and password file <none>
I20260806 02:13:21.689455 29205 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:21.689517 29205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:21.689800 29205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:21.691927 29205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "2ea2e0f855ab49d0b6e1e1ff32c77cb4"
format_stamp: "Formatted at 2026-08-06 02:13:21 on dist-test-slave-6k22"
server_key: "3c8abafeb25deaece3631e31a7c620d6"
server_key_iv: "c43073aaea4e8f4f5a608470bf4edeee"
server_key_version: "encryptionkey@0"
I20260806 02:13:21.692498 29205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "2ea2e0f855ab49d0b6e1e1ff32c77cb4"
format_stamp: "Formatted at 2026-08-06 02:13:21 on dist-test-slave-6k22"
server_key: "3c8abafeb25deaece3631e31a7c620d6"
server_key_iv: "c43073aaea4e8f4f5a608470bf4edeee"
server_key_version: "encryptionkey@0"
I20260806 02:13:21.696230 29205 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 02:13:21.698632 29219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:21.699883 29205 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:13:21.700012 29205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "2ea2e0f855ab49d0b6e1e1ff32c77cb4"
format_stamp: "Formatted at 2026-08-06 02:13:21 on dist-test-slave-6k22"
server_key: "3c8abafeb25deaece3631e31a7c620d6"
server_key_iv: "c43073aaea4e8f4f5a608470bf4edeee"
server_key_version: "encryptionkey@0"
I20260806 02:13:21.700124 29205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:21.704550 29205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:21.705111 29205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:21.705291 29205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:21.705817 29205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:21.706673 29205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:21.706750 29205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:21.706820 29205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:21.706856 29205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:21.715672 29205 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:42427
I20260806 02:13:21.715715 29338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:42427 every 8 connection(s)
I20260806 02:13:21.716545 29205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:13:21.718621 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 29205
I20260806 02:13:21.718729 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:13:21.718993 18713 external_mini_cluster.cc:1428] Setting key 16a090d49877c0c6c949341b8dec0afc
I20260806 02:13:21.722595 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:21.718220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:59413 (local address 127.18.70.126:33227)
0806 02:13:21.718392 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:21.718397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:21.719584 (+ 1187us) server_negotiation.cc:407] Connection header received
0806 02:13:21.720999 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:21.721002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:21.721046 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:21.721123 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:21.722110 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:21.722113 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:21.722116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:13:21.722128 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:21.722149 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:13:21.722166 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:21.722264 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:21.722422 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 02:13:21.722468 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 02:13:21.723382 29341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:21.718623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:33227 (local address 127.18.70.67:59413)
0806 02:13:21.719366 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:21.719415 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:21.720780 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:21.721282 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:21.721288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:21.721372 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:21.721866 (+ 494us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:13:21.721867 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:21.721905 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:13:21.721968 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:13:21.722256 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:13:21.722266 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:13:21.722275 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 02:13:21.722453 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 02:13:21.722680 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":161,"threads_started":1}
I20260806 02:13:21.724416 29339 heartbeater.cc:347] Connected to a master server at 127.18.70.126:33227
I20260806 02:13:21.724609 29339 heartbeater.cc:464] Registering TS with master...
I20260806 02:13:21.724949 29339 heartbeater.cc:511] Master 127.18.70.126:33227 requested a full tablet report, sending...
I20260806 02:13:21.725878 28866 ts_manager.cc:194] Registered new tserver with Master: 2ea2e0f855ab49d0b6e1e1ff32c77cb4 (127.18.70.67:42427)
I20260806 02:13:21.732739 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 02:13:21.735700 28907 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:13:21.734771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36992 (local address 127.18.70.126:33227)
0806 02:13:21.734902 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:21.734907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:21.735072 (+ 165us) server_negotiation.cc:407] Connection header received
0806 02:13:21.735205 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:21.735208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:21.735240 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:21.735347 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:21.735589 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36992: BlockingRecv error: recv got EOF from 127.0.0.1:36992 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 02:13:21.736059 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 28930
I20260806 02:13:21.744716 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 29069
I20260806 02:13:21.751626 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 29205
I20260806 02:13:21.759485 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 28834
2026-08-06T02:13:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (809 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:13:21 dist-test-slave-6k22 krb5kdc[29349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:13:21 dist-test-slave-6k22 krb5kdc[29349](info): set up 2 sockets
Aug 06 02:13:21 dist-test-slave-6k22 krb5kdc[29349](info): commencing operation
krb5kdc: starting...
W20260806 02:13:22.809715 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:13:22 dist-test-slave-6k22 krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:13:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:13:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:13:22.934903 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:37495
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:37495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:13:23.045982 29363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:23.046247 29363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:23.046286 29363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:23.050097 29363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:13:23.050146 29363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:23.050165 29363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:23.050180 29363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:13:23.050196 29363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:13:23.055284 29363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:37495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:37495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:23.056252 29363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:23.057235 29363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:23.066996 29368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:23.067004 29371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:23.067004 29369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:23.067247 29363 server_base.cc:1034] running on GCE node
I20260806 02:13:23.067863 29363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:23.068972 29363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:23.070137 29363 hybrid_clock.cc:648] HybridClock initialized: now 1785982403070110 us; error 43 us; skew 500 ppm
Aug 06 02:13:23 dist-test-slave-6k22 krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982403, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:13:23.077508 29363 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:13:23.078504 29363 webserver.cc:460] Webserver started at http://127.18.70.126:38567/ using document root <none> and password file <none>
I20260806 02:13:23.079088 29363 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:23.079151 29363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:23.079391 29363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:23.081528 29363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "5e740da107b6416fbada5820604af6e1"
format_stamp: "Formatted at 2026-08-06 02:13:23 on dist-test-slave-6k22"
server_key: "7e29b82cfeefc3675b3bcd812eb25afb"
server_key_iv: "a8dbc3d3f61ab59c4220399cac8f8f73"
server_key_version: "encryptionkey@0"
I20260806 02:13:23.082129 29363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "5e740da107b6416fbada5820604af6e1"
format_stamp: "Formatted at 2026-08-06 02:13:23 on dist-test-slave-6k22"
server_key: "7e29b82cfeefc3675b3bcd812eb25afb"
server_key_iv: "a8dbc3d3f61ab59c4220399cac8f8f73"
server_key_version: "encryptionkey@0"
I20260806 02:13:23.086122 29363 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 02:13:23.088546 29378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:23.089725 29363 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:13:23.089874 29363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "5e740da107b6416fbada5820604af6e1"
format_stamp: "Formatted at 2026-08-06 02:13:23 on dist-test-slave-6k22"
server_key: "7e29b82cfeefc3675b3bcd812eb25afb"
server_key_iv: "a8dbc3d3f61ab59c4220399cac8f8f73"
server_key_version: "encryptionkey@0"
I20260806 02:13:23.089995 29363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:23.099821 29363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:23.102434 29363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:23.102618 29363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:23.109737 29363 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:37495
I20260806 02:13:23.110001 29436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:37495 every 8 connection(s)
I20260806 02:13:23.110692 29363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:13:23.113386 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:13:23.118714 29437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1: Bootstrap starting.
I20260806 02:13:23.120906 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 29363
I20260806 02:13:23.121050 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:13:23.121228 29437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1: Neither blocks nor log segments found. Creating new log.
I20260806 02:13:23.121345 18713 external_mini_cluster.cc:1428] Setting key 54039206d4c5e94d7111e7ab049870d1
I20260806 02:13:23.122116 29437 log.cc:824] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1: Log is configured to *not* fsync() on all Append() calls
I20260806 02:13:23.124106 29437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1: No bootstrap required, opened a new log
I20260806 02:13:23.128360 29437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e740da107b6416fbada5820604af6e1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37495 } }
I20260806 02:13:23.128620 29437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:13:23.128708 29437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e740da107b6416fbada5820604af6e1, State: Initialized, Role: FOLLOWER
I20260806 02:13:23.129230 29437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e740da107b6416fbada5820604af6e1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37495 } }
I20260806 02:13:23.129446 29437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:13:23.129510 29437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:13:23.129603 29437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 02:13:23 dist-test-slave-6k22 krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:13:23.130543 29437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e740da107b6416fbada5820604af6e1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37495 } }
I20260806 02:13:23.130921 29437 leader_election.cc:302] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e740da107b6416fbada5820604af6e1; no voters:
I20260806 02:13:23.131290 29437 leader_election.cc:288] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:13:23.131413 29443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:13:23.131691 29443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [term 1 LEADER]: Becoming Leader. State: Replica: 5e740da107b6416fbada5820604af6e1, State: Running, Role: LEADER
I20260806 02:13:23.132071 29443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e740da107b6416fbada5820604af6e1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37495 } }
I20260806 02:13:23.132730 29437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:13:23.133785 29444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e740da107b6416fbada5820604af6e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e740da107b6416fbada5820604af6e1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37495 } } }
I20260806 02:13:23.133952 29444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [sys.catalog]: This master's current role is: LEADER
I20260806 02:13:23.134408 29452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:13:23.134392 29445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e740da107b6416fbada5820604af6e1. Latest consensus state: current_term: 1 leader_uuid: "5e740da107b6416fbada5820604af6e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e740da107b6416fbada5820604af6e1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 37495 } } }
I20260806 02:13:23.134564 29445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1 [sys.catalog]: This master's current role is: LEADER
I20260806 02:13:23.136912 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.122478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52504 (local address 127.18.70.126:37495)
0806 02:13:23.122906 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:23.122922 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:23.122956 (+ 34us) server_negotiation.cc:407] Connection header received
0806 02:13:23.123617 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:23.123645 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:23.123966 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:23.124320 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:23.125181 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.126135 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.127418 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.127958 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.128139 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.130860 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:23.130866 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:23.130883 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:13:23.130913 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:23.133071 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:23.133583 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:23.133593 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:23.133601 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:23.133771 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:23.134200 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:23.134203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:23.134205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:23.134803 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:23.135037 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:23.135503 (+ 466us) server_negotiation.cc:299] Negotiation successful
0806 02:13:23.135917 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":123,"threads_started":1}
I20260806 02:13:23.138237 29452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:13:23.144383 29452 catalog_manager.cc:1347] Generated new cluster ID: 7ccc89d1174940a5af114cf6f9c307b7
I20260806 02:13:23.144448 29452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:13:23.157191 29452 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:13:23.158052 29452 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:13:23.168300 29452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1: Generated new TSK 0
I20260806 02:13:23.168926 29452 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:13:23.200793 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37495
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:13:23.330780 29466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:23.331015 29466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:23.331058 29466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:23.334947 29466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:23.334995 29466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:23.335043 29466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:13:23.340044 29466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:23.340921 29466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:23.341822 29466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:23.350888 29472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:23.350888 29471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:23.351102 29466 server_base.cc:1034] running on GCE node
W20260806 02:13:23.351198 29474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:23.351651 29466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:23.352317 29466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:23.353526 29466 hybrid_clock.cc:648] HybridClock initialized: now 1785982403353491 us; error 44 us; skew 500 ppm
Aug 06 02:13:23 dist-test-slave-6k22 krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982403, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:13:23.356781 29466 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:13:23.357532 29466 webserver.cc:460] Webserver started at http://127.18.70.65:40137/ using document root <none> and password file <none>
I20260806 02:13:23.358038 29466 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:23.358091 29466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:23.358340 29466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:23.360337 29466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "5de67b99fd414ff7ad4795a94f6866fd"
format_stamp: "Formatted at 2026-08-06 02:13:23 on dist-test-slave-6k22"
server_key: "9069547f3cd137c45404382bea03d759"
server_key_iv: "18134e1bf8b83c651526938ca574ef43"
server_key_version: "encryptionkey@0"
I20260806 02:13:23.360824 29466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "5de67b99fd414ff7ad4795a94f6866fd"
format_stamp: "Formatted at 2026-08-06 02:13:23 on dist-test-slave-6k22"
server_key: "9069547f3cd137c45404382bea03d759"
server_key_iv: "18134e1bf8b83c651526938ca574ef43"
server_key_version: "encryptionkey@0"
I20260806 02:13:23.364379 29466 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 02:13:23.366695 29481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:23.367944 29466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:13:23.368036 29466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "5de67b99fd414ff7ad4795a94f6866fd"
format_stamp: "Formatted at 2026-08-06 02:13:23 on dist-test-slave-6k22"
server_key: "9069547f3cd137c45404382bea03d759"
server_key_iv: "18134e1bf8b83c651526938ca574ef43"
server_key_version: "encryptionkey@0"
I20260806 02:13:23.368112 29466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:23.392880 29466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:23.396153 29466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:23.396309 29466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:23.396871 29466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:23.397725 29466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:23.397780 29466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:23.397820 29466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:23.397836 29466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:23.407183 29466 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:35421
I20260806 02:13:23.407234 29602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:35421 every 8 connection(s)
I20260806 02:13:23.408162 29466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
Aug 06 02:13:23 dist-test-slave-6k22 krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982403, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:13:23.417667 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 29466
I20260806 02:13:23.417778 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:13:23.418076 18713 external_mini_cluster.cc:1428] Setting key ba437e5516fb1dee7e2e1201c029fd73
I20260806 02:13:23.422186 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.409979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:50255 (local address 127.18.70.126:37495)
0806 02:13:23.410124 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:23.410128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:23.411003 (+ 875us) server_negotiation.cc:407] Connection header received
0806 02:13:23.412417 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:23.412420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:23.412466 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:23.412542 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:23.413684 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.414206 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.415293 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.415499 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.415632 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.418265 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:23.418276 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:23.418278 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:13:23.418301 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:23.420095 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:23.420588 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:23.420594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:23.420599 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:23.420700 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:23.421025 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:23.421031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:23.421036 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:23.421292 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:23.421411 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:23.421895 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 02:13:23.422019 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 02:13:23.423013 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.410388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:37495 (local address 127.18.70.65:50255)
0806 02:13:23.410840 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:23.410888 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:23.412216 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:23.412696 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:23.412703 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:23.413055 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:23.413548 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:23.413561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.414311 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.414314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:23.415181 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:23.415190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.415672 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.415675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:23.415821 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.416445 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:23.416462 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:23.418081 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:13:23.420220 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:13:23.420224 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:13:23.420236 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:13:23.420486 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:13:23.420800 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:13:23.420802 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:13:23.420804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:13:23.420931 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:13:23.421398 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:13:23.421403 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:13:23.421618 (+ 215us) client_negotiation.cc:769] Sending connection context
0806 02:13:23.421832 (+ 214us) client_negotiation.cc:240] Negotiation successful
0806 02:13:23.422151 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":114,"threads_started":1}
I20260806 02:13:23.424337 29603 heartbeater.cc:347] Connected to a master server at 127.18.70.126:37495
I20260806 02:13:23.424598 29603 heartbeater.cc:464] Registering TS with master...
I20260806 02:13:23.425092 29603 heartbeater.cc:511] Master 127.18.70.126:37495 requested a full tablet report, sending...
I20260806 02:13:23.426831 29398 ts_manager.cc:194] Registered new tserver with Master: 5de67b99fd414ff7ad4795a94f6866fd (127.18.70.65:35421)
I20260806 02:13:23.428530 29398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:50255
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:13:23.446758 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37495
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:13:23.558842 29608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:23.559073 29608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:23.559114 29608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:23.563068 29608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:23.563119 29608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:23.563172 29608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:13:23.568284 29608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:23.569191 29608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:23.570134 29608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:23.579665 29613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:23.579665 29616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:23.579807 29608 server_base.cc:1034] running on GCE node
W20260806 02:13:23.579712 29614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:23.580355 29608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:23.581058 29608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:23.582252 29608 hybrid_clock.cc:648] HybridClock initialized: now 1785982403582217 us; error 48 us; skew 500 ppm
Aug 06 02:13:23 dist-test-slave-6k22 krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982403, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:13:23.584954 29608 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:13:23.585875 29608 webserver.cc:460] Webserver started at http://127.18.70.66:38109/ using document root <none> and password file <none>
I20260806 02:13:23.586454 29608 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:23.586547 29608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:23.586787 29608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:23.588953 29608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "6d7baed75f6142749b938ed304647119"
format_stamp: "Formatted at 2026-08-06 02:13:23 on dist-test-slave-6k22"
server_key: "7847357aac19111657285a7168395a23"
server_key_iv: "fc369e37d707a0823cc7cb668be6aae9"
server_key_version: "encryptionkey@0"
I20260806 02:13:23.589514 29608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "6d7baed75f6142749b938ed304647119"
format_stamp: "Formatted at 2026-08-06 02:13:23 on dist-test-slave-6k22"
server_key: "7847357aac19111657285a7168395a23"
server_key_iv: "fc369e37d707a0823cc7cb668be6aae9"
server_key_version: "encryptionkey@0"
I20260806 02:13:23.593326 29608 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 02:13:23.595885 29623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:23.597059 29608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:13:23.597201 29608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "6d7baed75f6142749b938ed304647119"
format_stamp: "Formatted at 2026-08-06 02:13:23 on dist-test-slave-6k22"
server_key: "7847357aac19111657285a7168395a23"
server_key_iv: "fc369e37d707a0823cc7cb668be6aae9"
server_key_version: "encryptionkey@0"
I20260806 02:13:23.597311 29608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:23.610076 29608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:23.612951 29608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:23.613143 29608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:23.613711 29608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:23.614622 29608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:23.614707 29608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:23.614777 29608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:23.614838 29608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:23.623872 29608 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:37579
I20260806 02:13:23.623900 29740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:37579 every 8 connection(s)
I20260806 02:13:23.624755 29608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:13:23.633301 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 29608
I20260806 02:13:23.633455 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:13:23.633728 18713 external_mini_cluster.cc:1428] Setting key 526d1f5086333b3c7d02705b42137009
Aug 06 02:13:23 dist-test-slave-6k22 krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982403, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:13:23.639809 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.626469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:59803 (local address 127.18.70.126:37495)
0806 02:13:23.626604 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:23.626607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:23.627426 (+ 819us) server_negotiation.cc:407] Connection header received
0806 02:13:23.628858 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:23.628861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:23.628919 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:23.629025 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:23.630265 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.630808 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.632023 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.632253 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.632375 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.635406 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:23.635411 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:23.635413 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:13:23.635440 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:23.637236 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:23.637948 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:23.637953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:23.637958 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:23.638061 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:23.638515 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:23.638520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:23.638523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:23.638798 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:23.638917 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:23.639427 (+ 510us) server_negotiation.cc:299] Negotiation successful
0806 02:13:23.639622 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 02:13:23.640609 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.626845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:37495 (local address 127.18.70.66:59803)
0806 02:13:23.627286 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:23.627327 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:23.628638 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:23.629210 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:23.629216 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:23.629640 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:23.630125 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:23.630138 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.630935 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.630938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:23.631900 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:23.631910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.632420 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.632424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:23.632573 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.633129 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:23.633150 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:23.635260 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:13:23.637373 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:13:23.637383 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:13:23.637400 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:13:23.637824 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:13:23.638174 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:13:23.638180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:13:23.638183 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:13:23.638412 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:13:23.638931 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:13:23.638937 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:13:23.639170 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 02:13:23.639375 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 02:13:23.639757 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":110,"threads_started":1}
I20260806 02:13:23.641950 29742 heartbeater.cc:347] Connected to a master server at 127.18.70.126:37495
I20260806 02:13:23.642225 29742 heartbeater.cc:464] Registering TS with master...
I20260806 02:13:23.642882 29742 heartbeater.cc:511] Master 127.18.70.126:37495 requested a full tablet report, sending...
I20260806 02:13:23.644160 29398 ts_manager.cc:194] Registered new tserver with Master: 6d7baed75f6142749b938ed304647119 (127.18.70.66:37579)
I20260806 02:13:23.644752 29398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:59803
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:13:23.663067 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37495
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:13:23.776711 29747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:13:23.776994 29747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:13:23.777068 29747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:13:23.781092 29747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:13:23.781163 29747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:13:23.781244 29747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:13:23.786422 29747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:37495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:13:23.787458 29747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:13:23.788509 29747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:13:23.798100 29754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:23.798094 29753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:13:23.798094 29756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:13:23.798386 29747 server_base.cc:1034] running on GCE node
I20260806 02:13:23.798841 29747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:13:23.799528 29747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:13:23.800757 29747 hybrid_clock.cc:648] HybridClock initialized: now 1785982403800722 us; error 46 us; skew 500 ppm
Aug 06 02:13:23 dist-test-slave-6k22 krb5kdc[29349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982403, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:13:23.803649 29747 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:13:23.804522 29747 webserver.cc:460] Webserver started at http://127.18.70.67:44577/ using document root <none> and password file <none>
I20260806 02:13:23.805100 29747 fs_manager.cc:359] Metadata directory not provided
I20260806 02:13:23.805197 29747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:13:23.805452 29747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:13:23.807533 29747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "c639e4f92e1b4a6d8f0049d14e2228db"
format_stamp: "Formatted at 2026-08-06 02:13:23 on dist-test-slave-6k22"
server_key: "ff3c49ec7acd2916bcabb097dfa3c24a"
server_key_iv: "5926d4467dc7d27f8971978b1279871a"
server_key_version: "encryptionkey@0"
I20260806 02:13:23.808184 29747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "c639e4f92e1b4a6d8f0049d14e2228db"
format_stamp: "Formatted at 2026-08-06 02:13:23 on dist-test-slave-6k22"
server_key: "ff3c49ec7acd2916bcabb097dfa3c24a"
server_key_iv: "5926d4467dc7d27f8971978b1279871a"
server_key_version: "encryptionkey@0"
I20260806 02:13:23.811851 29747 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 02:13:23.814168 29764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:23.815371 29747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:13:23.815549 29747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "c639e4f92e1b4a6d8f0049d14e2228db"
format_stamp: "Formatted at 2026-08-06 02:13:23 on dist-test-slave-6k22"
server_key: "ff3c49ec7acd2916bcabb097dfa3c24a"
server_key_iv: "5926d4467dc7d27f8971978b1279871a"
server_key_version: "encryptionkey@0"
I20260806 02:13:23.815693 29747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:13:23.840546 29747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:13:23.843932 29747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:13:23.844089 29747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:13:23.844659 29747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:13:23.845487 29747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:13:23.845536 29747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:23.845577 29747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:13:23.845592 29747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:13:23.855664 29747 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:42269
I20260806 02:13:23.856945 29747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:13:23.858641 29885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:42269 every 8 connection(s)
I20260806 02:13:23.859421 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 29747
I20260806 02:13:23.859606 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:13:23.859939 18713 external_mini_cluster.cc:1428] Setting key d51663c650e7033c96819abdf589e860
Aug 06 02:13:23 dist-test-slave-6k22 krb5kdc[29349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982403, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:13:23.874779 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.860735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:46693 (local address 127.18.70.126:37495)
0806 02:13:23.860903 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:23.860909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:23.862059 (+ 1150us) server_negotiation.cc:407] Connection header received
0806 02:13:23.863530 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:23.863535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:23.863625 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:23.863730 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:23.864980 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.865982 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.867197 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.867664 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.867797 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.870477 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:23.870483 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:23.870488 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:13:23.870524 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:23.872364 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:23.872999 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:23.873005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:23.873010 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:23.873116 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:23.873595 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:23.873599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:23.873600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:23.873820 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:23.873929 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:23.874494 (+ 565us) server_negotiation.cc:299] Negotiation successful
0806 02:13:23.874617 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 02:13:23.875617 29888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.861334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:37495 (local address 127.18.70.67:46693)
0806 02:13:23.861913 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:23.861969 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:23.863298 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:23.863878 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:23.863885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:23.864356 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:13:23.864835 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:23.864848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.866148 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.866151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:23.867071 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:23.867081 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.867813 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.867817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:23.867954 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.868584 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:13:23.868600 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:13:23.870298 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:13:23.872496 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:13:23.872502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:13:23.872516 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:13:23.872854 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:13:23.873250 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:13:23.873256 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:13:23.873261 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:13:23.873463 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:13:23.873944 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:13:23.873952 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:13:23.874227 (+ 275us) client_negotiation.cc:769] Sending connection context
0806 02:13:23.874453 (+ 226us) client_negotiation.cc:240] Negotiation successful
0806 02:13:23.874768 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":139,"threads_started":1}
I20260806 02:13:23.877148 29886 heartbeater.cc:347] Connected to a master server at 127.18.70.126:37495
I20260806 02:13:23.877394 29886 heartbeater.cc:464] Registering TS with master...
I20260806 02:13:23.877869 29886 heartbeater.cc:511] Master 127.18.70.126:37495 requested a full tablet report, sending...
I20260806 02:13:23.879000 29398 ts_manager.cc:194] Registered new tserver with Master: c639e4f92e1b4a6d8f0049d14e2228db (127.18.70.67:42269)
I20260806 02:13:23.879916 29398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:46693
I20260806 02:13:23.885892 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:13:23.896459 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.887890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52516 (local address 127.18.70.126:37495)
0806 02:13:23.888040 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:23.888047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:23.888180 (+ 133us) server_negotiation.cc:407] Connection header received
0806 02:13:23.888264 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:23.888269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:23.888326 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:23.888425 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:13:23.889050 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.889964 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.890776 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.891231 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.891438 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.892650 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:13:23.892657 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:13:23.892664 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:13:23.892699 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:13:23.894574 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:23.895002 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:23.895008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:23.895012 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:23.895117 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:13:23.895377 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:13:23.895382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:13:23.895386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:13:23.895751 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:13:23.895928 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:23.896078 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 02:13:23.896216 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 02:13:23.899665 29398 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:13:23.905296 29398 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:13:23.921811 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.914628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35421 (local address 127.0.0.1:51758)
0806 02:13:23.915781 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:23.915799 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:23.915897 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:23.916717 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:23.916723 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:23.916746 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:13:23.916885 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:23.916892 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.918396 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.918399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:23.919941 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:23.919951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.921221 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.921225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:23.921368 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.921413 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 02:13:23.921515 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 02:13:23.921670 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1067,"mutex_wait_us":152,"thread_start_us":93,"threads_started":1}
I20260806 02:13:23.921921 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.915722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51758 (local address 127.18.70.65:35421)
0806 02:13:23.915984 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:23.916008 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:23.916032 (+ 24us) server_negotiation.cc:407] Connection header received
0806 02:13:23.916110 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:23.916116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:23.916334 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:23.916507 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:13:23.917032 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.918218 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.920073 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.920907 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.921008 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.921140 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:23.921681 (+ 541us) server_negotiation.cc:299] Negotiation successful
0806 02:13:23.921774 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260806 02:13:23.923647 29898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.914506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:37579 (local address 127.0.0.1:45078)
0806 02:13:23.915302 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:23.915356 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:23.915476 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:23.917056 (+ 1580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:23.917059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:23.917078 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:13:23.917176 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:23.917182 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.918731 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.918737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:23.920524 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:23.920540 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.923051 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.923056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:23.923221 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.923256 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 02:13:23.923345 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 02:13:23.923440 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":90,"threads_started":1}
I20260806 02:13:23.924196 29900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.915197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.18.70.66:37579)
0806 02:13:23.916606 (+ 1409us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:23.916618 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:23.916640 (+ 22us) server_negotiation.cc:407] Connection header received
0806 02:13:23.916716 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:23.916721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:23.916899 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:23.917338 (+ 439us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:13:23.917504 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.918590 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.920701 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.922661 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.922789 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.922970 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:23.923850 (+ 880us) server_negotiation.cc:299] Negotiation successful
0806 02:13:23.923980 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1304,"thread_start_us":114,"threads_started":1}
I20260806 02:13:23.925029 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.916958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.18.70.67:42269)
0806 02:13:23.919491 (+ 2533us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:23.919502 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:23.919523 (+ 21us) server_negotiation.cc:407] Connection header received
0806 02:13:23.919624 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:23.919629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:23.919787 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:23.919932 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:13:23.920330 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.921377 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.923420 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.924338 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.924446 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.924570 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:23.924793 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 02:13:23.924886 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2433,"thread_start_us":104,"threads_started":1}
I20260806 02:13:23.925326 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:23.916493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:42269 (local address 127.0.0.1:46848)
0806 02:13:23.918825 (+ 2332us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:23.918839 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:23.918925 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:23.920093 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:23.920095 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:23.920112 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:13:23.920207 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:23.920212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.921522 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.921524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:23.923277 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:23.923288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:23.924476 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:23.924482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:23.924624 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:23.924662 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 02:13:23.924753 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 02:13:23.924845 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2267,"thread_start_us":111,"threads_started":1}
I20260806 02:13:23.925276 29533 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe2cc9b26c3410d969b48edc667cb19 (DEFAULT_TABLE table=test-table [id=f33d4c0a86074bebb66e7f8fd067dabf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:13:23.926478 29533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe2cc9b26c3410d969b48edc667cb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:13:23.928248 29816 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe2cc9b26c3410d969b48edc667cb19 (DEFAULT_TABLE table=test-table [id=f33d4c0a86074bebb66e7f8fd067dabf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:13:23.929276 29816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe2cc9b26c3410d969b48edc667cb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:13:23.930389 29673 tablet_service.cc:1467] Processing CreateTablet for tablet 8fe2cc9b26c3410d969b48edc667cb19 (DEFAULT_TABLE table=test-table [id=f33d4c0a86074bebb66e7f8fd067dabf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:13:23.931519 29673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fe2cc9b26c3410d969b48edc667cb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:13:23.937089 29904 tablet_bootstrap.cc:492] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd: Bootstrap starting.
I20260806 02:13:23.938686 29905 tablet_bootstrap.cc:492] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db: Bootstrap starting.
I20260806 02:13:23.939641 29904 tablet_bootstrap.cc:654] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd: Neither blocks nor log segments found. Creating new log.
I20260806 02:13:23.940841 29905 tablet_bootstrap.cc:654] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db: Neither blocks nor log segments found. Creating new log.
I20260806 02:13:23.941073 29904 log.cc:824] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd: Log is configured to *not* fsync() on all Append() calls
I20260806 02:13:23.941795 29906 tablet_bootstrap.cc:492] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119: Bootstrap starting.
I20260806 02:13:23.942118 29905 log.cc:824] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db: Log is configured to *not* fsync() on all Append() calls
I20260806 02:13:23.943400 29904 tablet_bootstrap.cc:492] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd: No bootstrap required, opened a new log
I20260806 02:13:23.943643 29904 ts_tablet_manager.cc:1397] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 02:13:23.944674 29905 tablet_bootstrap.cc:492] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db: No bootstrap required, opened a new log
I20260806 02:13:23.944888 29905 ts_tablet_manager.cc:1397] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 02:13:23.946134 29906 tablet_bootstrap.cc:654] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119: Neither blocks nor log segments found. Creating new log.
I20260806 02:13:23.947013 29906 log.cc:824] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119: Log is configured to *not* fsync() on all Append() calls
I20260806 02:13:23.948300 29904 raft_consensus.cc:348] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7baed75f6142749b938ed304647119" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37579 } } peers { permanent_uuid: "5de67b99fd414ff7ad4795a94f6866fd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35421 } } peers { permanent_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42269 } }
I20260806 02:13:23.948582 29904 raft_consensus.cc:374] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:13:23.948640 29904 raft_consensus.cc:729] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de67b99fd414ff7ad4795a94f6866fd, State: Initialized, Role: FOLLOWER
I20260806 02:13:23.948993 29906 tablet_bootstrap.cc:492] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119: No bootstrap required, opened a new log
I20260806 02:13:23.949147 29906 ts_tablet_manager.cc:1397] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 02:13:23.949137 29904 consensus_queue.cc:260] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7baed75f6142749b938ed304647119" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37579 } } peers { permanent_uuid: "5de67b99fd414ff7ad4795a94f6866fd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35421 } } peers { permanent_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42269 } }
I20260806 02:13:23.950098 29904 ts_tablet_manager.cc:1428] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 02:13:23.949990 29905 raft_consensus.cc:348] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7baed75f6142749b938ed304647119" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37579 } } peers { permanent_uuid: "5de67b99fd414ff7ad4795a94f6866fd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35421 } } peers { permanent_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42269 } }
I20260806 02:13:23.950270 29905 raft_consensus.cc:374] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:13:23.950325 29905 raft_consensus.cc:729] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c639e4f92e1b4a6d8f0049d14e2228db, State: Initialized, Role: FOLLOWER
I20260806 02:13:23.950423 29603 heartbeater.cc:503] Master 127.18.70.126:37495 was elected leader, sending a full tablet report...
I20260806 02:13:23.951184 29905 consensus_queue.cc:260] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7baed75f6142749b938ed304647119" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37579 } } peers { permanent_uuid: "5de67b99fd414ff7ad4795a94f6866fd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35421 } } peers { permanent_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42269 } }
I20260806 02:13:23.952314 29905 ts_tablet_manager.cc:1428] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 02:13:23.953492 29886 heartbeater.cc:503] Master 127.18.70.126:37495 was elected leader, sending a full tablet report...
I20260806 02:13:23.953639 29906 raft_consensus.cc:348] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7baed75f6142749b938ed304647119" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37579 } } peers { permanent_uuid: "5de67b99fd414ff7ad4795a94f6866fd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35421 } } peers { permanent_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42269 } }
I20260806 02:13:23.953929 29906 raft_consensus.cc:374] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:13:23.954022 29906 raft_consensus.cc:729] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7baed75f6142749b938ed304647119, State: Initialized, Role: FOLLOWER
I20260806 02:13:23.954486 29906 consensus_queue.cc:260] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7baed75f6142749b938ed304647119" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37579 } } peers { permanent_uuid: "5de67b99fd414ff7ad4795a94f6866fd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35421 } } peers { permanent_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42269 } }
I20260806 02:13:23.955538 29906 ts_tablet_manager.cc:1428] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:13:23.955938 29742 heartbeater.cc:503] Master 127.18.70.126:37495 was elected leader, sending a full tablet report...
W20260806 02:13:24.109427 29887 tablet.cc:2369] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 02:13:24.126072 29743 tablet.cc:2369] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:13:24.158813 29911 raft_consensus.cc:482] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:13:24.159066 29911 raft_consensus.cc:504] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7baed75f6142749b938ed304647119" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37579 } } peers { permanent_uuid: "5de67b99fd414ff7ad4795a94f6866fd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35421 } } peers { permanent_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42269 } }
W20260806 02:13:24.159480 29604 tablet.cc:2369] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:13:24.160517 29911 leader_election.cc:288] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d7baed75f6142749b938ed304647119 (127.18.70.66:37579), 5de67b99fd414ff7ad4795a94f6866fd (127.18.70.65:35421)
I20260806 02:13:24.164017 29888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:24.160740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35421 (local address 127.18.70.67:43697)
0806 02:13:24.160865 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:24.160880 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:24.160986 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:24.161252 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:24.161255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:24.161274 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:13:24.161395 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:24.161400 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:24.162110 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:24.162114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:24.162907 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:24.162916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:24.163593 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:24.163597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:24.163706 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:24.163725 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 02:13:24.163819 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 02:13:24.163892 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 02:13:24.164017 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:24.160890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:43697 (local address 127.18.70.65:35421)
0806 02:13:24.161042 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:24.161046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:24.161058 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:13:24.161103 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:24.161105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:24.161144 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:24.161239 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:13:24.161506 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:24.162002 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:24.163045 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:24.163449 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:24.163593 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:24.163648 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:24.163834 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 02:13:24.163900 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 02:13:24.164685 29553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe2cc9b26c3410d969b48edc667cb19" candidate_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de67b99fd414ff7ad4795a94f6866fd" is_pre_election: true
I20260806 02:13:24.164872 29900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:24.160834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:46939 (local address 127.18.70.66:37579)
0806 02:13:24.160986 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:24.160991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:24.161488 (+ 497us) server_negotiation.cc:407] Connection header received
0806 02:13:24.161640 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:24.161643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:24.161689 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:24.161770 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:13:24.162069 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:24.162563 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:24.163594 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:24.163969 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:24.164115 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:24.164182 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:24.164681 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 02:13:24.164750 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:13:24.164875 29914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:24.160748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:37579 (local address 127.18.70.67:46939)
0806 02:13:24.161381 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:24.161393 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:13:24.161520 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:13:24.161790 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:13:24.161793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:13:24.161808 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:13:24.161947 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:24.161952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:24.162679 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:24.162683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:24.163447 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:13:24.163456 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:13:24.164445 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:13:24.164449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:13:24.164544 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:24.164566 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 02:13:24.164652 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 02:13:24.164729 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":297,"thread_start_us":132,"threads_started":1}
I20260806 02:13:24.165009 29553 raft_consensus.cc:2454] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c639e4f92e1b4a6d8f0049d14e2228db in term 0.
I20260806 02:13:24.165544 29768 leader_election.cc:302] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de67b99fd414ff7ad4795a94f6866fd, c639e4f92e1b4a6d8f0049d14e2228db; no voters:
I20260806 02:13:24.165547 29694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe2cc9b26c3410d969b48edc667cb19" candidate_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d7baed75f6142749b938ed304647119" is_pre_election: true
I20260806 02:13:24.165836 29694 raft_consensus.cc:2454] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c639e4f92e1b4a6d8f0049d14e2228db in term 0.
I20260806 02:13:24.165865 29911 raft_consensus.cc:2781] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:13:24.165939 29911 raft_consensus.cc:482] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:13:24.166008 29911 raft_consensus.cc:3037] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:24.167152 29911 raft_consensus.cc:504] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7baed75f6142749b938ed304647119" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37579 } } peers { permanent_uuid: "5de67b99fd414ff7ad4795a94f6866fd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35421 } } peers { permanent_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42269 } }
I20260806 02:13:24.167529 29911 leader_election.cc:288] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [CANDIDATE]: Term 1 election: Requested vote from peers 6d7baed75f6142749b938ed304647119 (127.18.70.66:37579), 5de67b99fd414ff7ad4795a94f6866fd (127.18.70.65:35421)
I20260806 02:13:24.167845 29694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe2cc9b26c3410d969b48edc667cb19" candidate_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d7baed75f6142749b938ed304647119"
I20260806 02:13:24.168006 29694 raft_consensus.cc:3037] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:24.168001 29553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fe2cc9b26c3410d969b48edc667cb19" candidate_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de67b99fd414ff7ad4795a94f6866fd"
I20260806 02:13:24.168144 29553 raft_consensus.cc:3037] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:13:24.169131 29694 raft_consensus.cc:2454] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c639e4f92e1b4a6d8f0049d14e2228db in term 1.
I20260806 02:13:24.169319 29553 raft_consensus.cc:2454] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c639e4f92e1b4a6d8f0049d14e2228db in term 1.
I20260806 02:13:24.169472 29765 leader_election.cc:302] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d7baed75f6142749b938ed304647119, c639e4f92e1b4a6d8f0049d14e2228db; no voters:
I20260806 02:13:24.169716 29911 raft_consensus.cc:2781] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:13:24.169935 29911 raft_consensus.cc:686] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [term 1 LEADER]: Becoming Leader. State: Replica: c639e4f92e1b4a6d8f0049d14e2228db, State: Running, Role: LEADER
I20260806 02:13:24.170243 29911 consensus_queue.cc:237] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7baed75f6142749b938ed304647119" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37579 } } peers { permanent_uuid: "5de67b99fd414ff7ad4795a94f6866fd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35421 } } peers { permanent_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42269 } }
I20260806 02:13:24.173622 29398 catalog_manager.cc:5495] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db reported cstate change: term changed from 0 to 1, leader changed from <none> to c639e4f92e1b4a6d8f0049d14e2228db (127.18.70.67). New cstate: current_term: 1 leader_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7baed75f6142749b938ed304647119" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5de67b99fd414ff7ad4795a94f6866fd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c639e4f92e1b4a6d8f0049d14e2228db" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 42269 } health_report { overall_health: HEALTHY } } }
I20260806 02:13:24.210381 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:24.206616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.18.70.67:42269)
0806 02:13:24.206776 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 02:13:24.206780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:13:24.206795 (+ 15us) server_negotiation.cc:407] Connection header received
0806 02:13:24.206959 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:13:24.206962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:13:24.207018 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:13:24.207131 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:13:24.207383 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:24.207955 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:24.208860 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:13:24.209104 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:13:24.209226 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:13:24.209563 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:13:24.209965 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:13:24.210128 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 02:13:24.210167 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 02:13:24.210232 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 02:13:24.216295 29694 raft_consensus.cc:1261] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119 [term 1 FOLLOWER]: Refusing update from remote peer c639e4f92e1b4a6d8f0049d14e2228db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:13:24.216333 29553 raft_consensus.cc:1261] T 8fe2cc9b26c3410d969b48edc667cb19 P 5de67b99fd414ff7ad4795a94f6866fd [term 1 FOLLOWER]: Refusing update from remote peer c639e4f92e1b4a6d8f0049d14e2228db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:13:24.217073 29915 consensus_queue.cc:1035] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d7baed75f6142749b938ed304647119" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:13:24.217326 29911 consensus_queue.cc:1035] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [LEADER]: Connected to new peer: Peer: permanent_uuid: "5de67b99fd414ff7ad4795a94f6866fd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:13:24.226874 29920 mvcc.cc:205] Tried to move back new op lower bound from 7315383927662706688 to 7315383927483711488. Current Snapshot: MvccSnapshot[applied={T|T < 7315383927662706688}]
I20260806 02:13:24.234083 29398 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:13:24.234216 29398 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:13:24.236662 29398 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e740da107b6416fbada5820604af6e1: Sending DeleteTablet for 3 replicas of tablet 8fe2cc9b26c3410d969b48edc667cb19
I20260806 02:13:24.237373 29673 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fe2cc9b26c3410d969b48edc667cb19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:13:24 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:45078
I20260806 02:13:24.237478 29533 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fe2cc9b26c3410d969b48edc667cb19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:13:24 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:51758
I20260806 02:13:24.238001 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 29466
I20260806 02:13:24.237979 29928 tablet_replica.cc:330] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119: stopping tablet replica
I20260806 02:13:24.239027 29816 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fe2cc9b26c3410d969b48edc667cb19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:13:24 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:46848
I20260806 02:13:24.239451 29930 tablet_replica.cc:330] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db: stopping tablet replica
I20260806 02:13:24.239886 29930 raft_consensus.cc:2229] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:13:24.240386 29930 raft_consensus.cc:2258] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:13:24.241590 29928 raft_consensus.cc:2229] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:13:24.241931 29928 raft_consensus.cc:2258] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:13:24.242162 29930 ts_tablet_manager.cc:1905] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:13:24.243435 29928 ts_tablet_manager.cc:1905] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:13:24.245728 29930 ts_tablet_manager.cc:1918] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:13:24.245904 29930 log.cc:1196] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/8fe2cc9b26c3410d969b48edc667cb19
I20260806 02:13:24.246115 29928 ts_tablet_manager.cc:1918] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:13:24.246217 29928 log.cc:1196] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/8fe2cc9b26c3410d969b48edc667cb19
I20260806 02:13:24.246428 29930 ts_tablet_manager.cc:1939] T 8fe2cc9b26c3410d969b48edc667cb19 P c639e4f92e1b4a6d8f0049d14e2228db: Deleting consensus metadata
I20260806 02:13:24.246521 29928 ts_tablet_manager.cc:1939] T 8fe2cc9b26c3410d969b48edc667cb19 P 6d7baed75f6142749b938ed304647119: Deleting consensus metadata
I20260806 02:13:24.247725 29380 catalog_manager.cc:4841] TS 6d7baed75f6142749b938ed304647119 (127.18.70.66:37579): tablet 8fe2cc9b26c3410d969b48edc667cb19 (table test-table [id=f33d4c0a86074bebb66e7f8fd067dabf]) successfully deleted
I20260806 02:13:24.247725 29384 catalog_manager.cc:4841] TS c639e4f92e1b4a6d8f0049d14e2228db (127.18.70.67:42269): tablet 8fe2cc9b26c3410d969b48edc667cb19 (table test-table [id=f33d4c0a86074bebb66e7f8fd067dabf]) successfully deleted
W20260806 02:13:24.251154 29384 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:35421 (error 108)
I20260806 02:13:24.251288 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 29608
I20260806 02:13:24.256101 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:13:24.255783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35421 (local address 127.0.0.1:51768)
0806 02:13:24.255937 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 02:13:24.256014 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:35421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 02:13:24.256269 29384 catalog_manager.cc:4584] TS 5de67b99fd414ff7ad4795a94f6866fd (127.18.70.65:35421): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8fe2cc9b26c3410d969b48edc667cb19: Network error: Client connection negotiation failed: client connection to 127.18.70.65:35421: connect: Connection refused (error 111)
I20260806 02:13:24.260737 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 29747
I20260806 02:13:24.267829 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 29363
2026-08-06T02:13:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2509 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:13:24 dist-test-slave-6k22 krb5kdc[29934](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:13:24 dist-test-slave-6k22 krb5kdc[29934](info): set up 2 sockets
Aug 06 02:13:24 dist-test-slave-6k22 krb5kdc[29934](info): commencing operation
krb5kdc: starting...
W20260806 02:13:25.329860 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:13:25 dist-test-slave-6k22 krb5kdc[29934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:13:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:13:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.70.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.70.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/rangeradmin_127.18.70.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/rangeradmin_127.18.70.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.70.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.70.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/rangerlookup_127.18.70.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/rangerlookup_127.18.70.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.70.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.70.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/HTTP_127.18.70.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/HTTP_127.18.70.84@KRBTEST.COM.keytab.
I20260806 02:13:25.495972 18713 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskVlzDnt/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/postgres -l logfile start
2026-08-06 02:13:27.973 UTC [29966] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 02:13:27.973 UTC [29966] LOG: listening on IPv4 address "127.18.70.84", port 47855
2026-08-06 02:13:27.976 UTC [29966] LOG: listening on Unix socket "/tmp/.s.PGSQL.47855"
2026-08-06 02:13:27.997 UTC [29969] LOG: database system was shut down at 2026-08-06 02:13:26 UTC
2026-08-06 02:13:28.001 UTC [29966] LOG: database system is ready to accept connections
I20260806 02:13:28.971503 18713 mini_postgres.cc:96] Postgres bound to 47855
2026-08-06 02:13:28.980 UTC [29978] FATAL: database "slave" does not exist
127.18.70.84:47855 - accepting connections
I20260806 02:13:28.981580 18713 mini_ranger.cc:161] Starting Ranger...
I20260806 02:13:28.998472 18713 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 02:13:29.451896 18713 mini_ranger.cc:87] Created ranger Postgres database
I20260806 02:13:29.452014 18713 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-admin
2026-08-06 02:13:29,785 [I] DB FLAVOR :POSTGRES
2026-08-06 02:13:29,785 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 02:13:29,785 [I] Checking connection..
2026-08-06 02:13:29,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 02:13:30,046 [I] Checking connection passed.
2026-08-06 02:13:30,046 [I] --------- Verifying version history table ---------
2026-08-06 02:13:30,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 02:13:30,321 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 02:13:30,321 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 02:13:30,584 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 02:13:30,584 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 02:13:30,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 02:13:30.842 UTC [30045] WARNING: there is no transaction in progress
2026-08-06 02:13:30,850 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 02:13:30,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 02:13:31,119 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 02:13:31,119 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 02:13:31,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 02:13:31,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 02:13:31,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6k22.c.gcp-upstream.internal','N') ;"
2026-08-06 02:13:31,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 02:13:32,082 [I] Table x_portal_user does not exist in database ranger
2026-08-06 02:13:32,082 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 02:13:32,338 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 02:13:32,338 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 02:13:32,594 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 02:13:32,838 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 02:13:32,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 02:13:33.763 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.779 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.795 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.810 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.825 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.841 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.895 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.901 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.914 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.924 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.934 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.947 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.957 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.962 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.969 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:33.975 UTC [30159] WARNING: there is no transaction in progress
2026-08-06 02:13:34,599 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 02:13:34,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 02:13:34,859 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 02:13:34,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 02:13:35,094 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 02:13:35,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 02:13:35,326 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 02:13:35,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 02:13:35,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 02:13:35,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6k22.c.gcp-upstream.internal';"
2026-08-06 02:13:36,052 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 02:13:36,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 02:13:36,313 [I] DB_PATCHES have already been applied
I20260806 02:13:36.319015 18713 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-admin:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskVlzDnt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 02:13:36.319079 18713 mini_ranger.cc:193] Using host: 127.18.70.84
I20260806 02:13:36.323755 18713 mini_ranger.cc:227] Ranger admin URL: http://127.18.70.84:50401
Aug 06, 2026 2:13:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 2:13:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 2:13:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 2:13:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 2:13:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 2:13:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 2:13:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 02:13:37.460289 18713 mini_ranger.cc:160] Time spent starting Ranger: real 8.479s user 0.004s sys 0.008s
Aug 06, 2026 2:13:57 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 2:13:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 2:13:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 2:13:57 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 2:13:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 2:13:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 2:13:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 2:13:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 2:13:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 2:13:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 2:13:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 2:13:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 2:13:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 2:13:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 2:13:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 2:13:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 2:13:57 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 2:13:58 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 02:13:59.489024 18713 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.70.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.70.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/rangerkms_127.18.70.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/rangerkms_127.18.70.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.70.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.70.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.70.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/HTTP_127.18.70.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.70.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/HTTP_127.18.70.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 02:13:59 dist-test-slave-6k22 krb5kdc[29934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982439, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 02:13:59.578943 18713 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 02:13:59.589910 18713 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 02:14:00.388841 18713 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 02:14:00.388955 18713 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-kms
I20260806 02:14:00.389065 18713 mini_ranger_kms.cc:227] Using postgres at 127.18.70.84:47855
2026-08-06 02:14:00,621 [I] DB FLAVOR :POSTGRES
2026-08-06 02:14:00,622 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 02:14:00,622 [I] Checking connection
2026-08-06 02:14:00,622 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 02:14:00,869 [I] connection success
2026-08-06 02:14:00,869 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 02:14:00,869 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 02:14:00,870 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 02:14:01,117 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 02:14:01,117 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 02:14:01,118 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 02:14:01,118 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:47855/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 02:14:01,401 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 02:14:01.840814 18713 mini_ranger_kms.cc:310] Created kms service
I20260806 02:14:02.012707 18713 mini_ranger_kms.cc:326] Created kudu user
I20260806 02:14:02.056171 18713 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 02:14:02 dist-test-slave-6k22 krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982439, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:02.472293 18713 mini_ranger_kms.cc:384] Added ranger policy
I20260806 02:14:02.472461 18713 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-kms:/tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 02:14:02.472491 18713 mini_ranger_kms.cc:240] Using host: 127.18.70.84
I20260806 02:14:02.477252 18713 mini_ranger_kms.cc:276] Ranger KMS PID: 30349
I20260806 02:14:02.477388 18713 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.70.84:52927
Aug 06, 2026 2:14:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 2:14:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 2:14:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 2:14:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 2:14:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 2:14:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 02:14:04.860394 18713 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.281s user 0.010s sys 0.210s
I20260806 02:14:04.860536 18713 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 02:14:04.860669 18713 mini_ranger_kms.cc:401] 127.18.70.84:52927/kms/v1/keys
Aug 06 02:14:04 dist-test-slave-6k22 krb5kdc[29934](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785982444, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.70.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 02:14:05 dist-test-slave-6k22 krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982444, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.70.84@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
Aug 06 02:14:05 dist-test-slave-6k22 krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982444, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.70.84@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
Aug 06 02:14:07 dist-test-slave-6k22 krb5kdc[29934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:14:07.934866 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:42121
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:42121
--ranger_config_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.70.84:52927/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:14:08.046372 30403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:14:08.046619 30403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:14:08.046656 30403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:14:08.050752 30403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:14:08.050813 30403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:14:08.050832 30403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:14:08.050869 30403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:14:08.050902 30403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:14:08.056471 30403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.70.84:52927/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:42121
--ranger_config_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:42121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:14:08.057513 30403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:14:08.058568 30403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:14:08.067344 30408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:08.067423 30409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:08.067368 30411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:08.067545 30403 server_base.cc:1034] running on GCE node
I20260806 02:14:08.068135 30403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:14:08.069130 30403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:14:08.070325 30403 hybrid_clock.cc:648] HybridClock initialized: now 1785982448070308 us; error 29 us; skew 500 ppm
Aug 06 02:14:08 dist-test-slave-6k22 krb5kdc[29934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982448, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:08.073155 30403 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:14:08.073964 30403 webserver.cc:460] Webserver started at http://127.18.70.126:43147/ using document root <none> and password file <none>
I20260806 02:14:08.074473 30403 fs_manager.cc:359] Metadata directory not provided
I20260806 02:14:08.074524 30403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:14:08.074681 30403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 02:14:08 dist-test-slave-6k22 krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982448, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:08.127486 30403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "ce560bed780f4689b4fc249eeffe34e7"
format_stamp: "Formatted at 2026-08-06 02:14:08 on dist-test-slave-6k22"
server_key: "5e3a6452d2306316b856001998c9955a"
server_key_iv: "1f46c3e0556942762305ed63028cefed"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:08.128285 30403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "ce560bed780f4689b4fc249eeffe34e7"
format_stamp: "Formatted at 2026-08-06 02:14:08 on dist-test-slave-6k22"
server_key: "5e3a6452d2306316b856001998c9955a"
server_key_iv: "1f46c3e0556942762305ed63028cefed"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:08.133430 30403 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 02:14:08.151795 30418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:08.153497 30403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 02:14:08.153641 30403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "ce560bed780f4689b4fc249eeffe34e7"
format_stamp: "Formatted at 2026-08-06 02:14:08 on dist-test-slave-6k22"
server_key: "5e3a6452d2306316b856001998c9955a"
server_key_iv: "1f46c3e0556942762305ed63028cefed"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:08.153807 30403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:14:08.169646 30403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:14:08.172906 30403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:14:08.173157 30403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:14:08.182600 30403 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:42121
I20260806 02:14:08.182607 30471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:42121 every 8 connection(s)
I20260806 02:14:08.183840 30403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:14:08.186893 30473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:14:08.192368 30473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7: Bootstrap starting.
I20260806 02:14:08.193528 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 30403
I20260806 02:14:08.193665 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:14:08.193925 18713 external_mini_cluster.cc:1428] Setting key 74104e78f81a493c927c2a33b2e3bf70
I20260806 02:14:08.195282 30473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7: Neither blocks nor log segments found. Creating new log.
I20260806 02:14:08.196388 30473 log.cc:824] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7: Log is configured to *not* fsync() on all Append() calls
I20260806 02:14:08.198834 30473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7: No bootstrap required, opened a new log
I20260806 02:14:08.203375 30473 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce560bed780f4689b4fc249eeffe34e7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42121 } }
I20260806 02:14:08.203749 30473 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:14:08.203869 30473 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce560bed780f4689b4fc249eeffe34e7, State: Initialized, Role: FOLLOWER
I20260806 02:14:08.204401 30473 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce560bed780f4689b4fc249eeffe34e7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42121 } }
I20260806 02:14:08.204568 30473 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:14:08.204658 30473 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:14:08.204805 30473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 02:14:08 dist-test-slave-6k22 krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:14:08.205928 30473 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce560bed780f4689b4fc249eeffe34e7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42121 } }
I20260806 02:14:08.206396 30473 leader_election.cc:302] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce560bed780f4689b4fc249eeffe34e7; no voters:
I20260806 02:14:08.206828 30473 leader_election.cc:288] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:14:08.207047 30478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:14:08.207311 30478 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [term 1 LEADER]: Becoming Leader. State: Replica: ce560bed780f4689b4fc249eeffe34e7, State: Running, Role: LEADER
I20260806 02:14:08.207724 30478 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce560bed780f4689b4fc249eeffe34e7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42121 } }
I20260806 02:14:08.208455 30473 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:14:08.209095 30478 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce560bed780f4689b4fc249eeffe34e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce560bed780f4689b4fc249eeffe34e7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42121 } } }
I20260806 02:14:08.209220 30479 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce560bed780f4689b4fc249eeffe34e7. Latest consensus state: current_term: 1 leader_uuid: "ce560bed780f4689b4fc249eeffe34e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce560bed780f4689b4fc249eeffe34e7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42121 } } }
I20260806 02:14:08.209350 30479 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [sys.catalog]: This master's current role is: LEADER
I20260806 02:14:08.209285 30478 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7 [sys.catalog]: This master's current role is: LEADER
I20260806 02:14:08.209738 30482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:14:08.212131 30476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:08.194980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37542 (local address 127.18.70.126:42121)
0806 02:14:08.195495 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:08.195513 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:08.195571 (+ 58us) server_negotiation.cc:407] Connection header received
0806 02:14:08.196528 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:08.196555 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:08.197022 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:08.197425 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:08.198384 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:08.199967 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:08.201412 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:08.202108 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:08.202300 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:08.205521 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:08.205530 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:08.205553 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:08.205588 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:08.207847 (+ 2259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:08.208436 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:08.208442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:08.208448 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:08.208634 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:08.209003 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:08.209006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:08.209008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:08.209617 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:08.209811 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:08.210696 (+ 885us) server_negotiation.cc:299] Negotiation successful
0806 02:14:08.211024 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":151,"threads_started":1}
I20260806 02:14:08.213304 30482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:14:08.213899 30473 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 02:14:08.229152 30482 catalog_manager.cc:1347] Generated new cluster ID: 48cebb2b95954469ba359ddac4f7a7b2
I20260806 02:14:08.229234 30482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:14:08.243166 30482 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:14:08.244671 30482 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:14:08.257951 30482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7: Generated new TSK 0
I20260806 02:14:08.258870 30482 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:14:08.932298 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.70.84:52927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42121
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:14:09.075505 30522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:14:09.075878 30522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:14:09.075940 30522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:14:09.081800 30522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:14:09.081961 30522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:14:09.082109 30522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:14:09.089913 30522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.70.84:52927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:14:09.091260 30522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:14:09.092644 30522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:14:09.104255 30529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:09.104245 30534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:09.104332 30530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:09.104831 30522 server_base.cc:1034] running on GCE node
I20260806 02:14:09.105221 30522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:14:09.105816 30522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:14:09.109098 30522 hybrid_clock.cc:648] HybridClock initialized: now 1785982449109057 us; error 47 us; skew 500 ppm
Aug 06 02:14:09 dist-test-slave-6k22 krb5kdc[29934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:09.112960 30522 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:14:09.114038 30522 webserver.cc:460] Webserver started at http://127.18.70.65:44541/ using document root <none> and password file <none>
I20260806 02:14:09.114836 30522 fs_manager.cc:359] Metadata directory not provided
I20260806 02:14:09.114928 30522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:14:09.115165 30522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 02:14:09 dist-test-slave-6k22 krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:09.131836 30522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "c87d226ec5aa44778d8309721e439a95"
format_stamp: "Formatted at 2026-08-06 02:14:09 on dist-test-slave-6k22"
server_key: "b678cdcc394d52369857a1964e8030e0"
server_key_iv: "0f7d0703e07a963c9833662cb94299d1"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:09.132503 30522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "c87d226ec5aa44778d8309721e439a95"
format_stamp: "Formatted at 2026-08-06 02:14:09 on dist-test-slave-6k22"
server_key: "b678cdcc394d52369857a1964e8030e0"
server_key_iv: "0f7d0703e07a963c9833662cb94299d1"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:09.136897 30522 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 02:14:09.158968 30541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:09.160674 30522 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 02:14:09.160799 30522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "c87d226ec5aa44778d8309721e439a95"
format_stamp: "Formatted at 2026-08-06 02:14:09 on dist-test-slave-6k22"
server_key: "b678cdcc394d52369857a1964e8030e0"
server_key_iv: "0f7d0703e07a963c9833662cb94299d1"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:09.160938 30522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:14:09.179728 30522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:14:09.182914 30522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:14:09.183164 30522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:14:09.183884 30522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:14:09.184955 30522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:14:09.185063 30522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:09.185200 30522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:14:09.185281 30522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:09.202522 30522 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:44495
I20260806 02:14:09.203783 30522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:14:09.205112 30663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:44495 every 8 connection(s)
I20260806 02:14:09.208842 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 30522
I20260806 02:14:09.208976 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:14:09.209306 18713 external_mini_cluster.cc:1428] Setting key 9c52e7e61367781cb27d8bbc64aa1aca
Aug 06 02:14:09 dist-test-slave-6k22 krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:14:09.223898 30667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:09.208821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:48727 (local address 127.18.70.126:42121)
0806 02:14:09.209666 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:09.209675 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:09.209695 (+ 20us) server_negotiation.cc:407] Connection header received
0806 02:14:09.211255 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:09.211260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:09.211383 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:09.211464 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:09.213142 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:09.214016 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:09.215750 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:09.216100 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:09.216186 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:09.219194 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:09.219200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:09.219204 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:09.219234 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:09.221336 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:09.221974 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:09.221981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:09.221985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:09.222092 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:09.222558 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:09.222562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:09.222563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:09.222774 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:09.222853 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:09.223551 (+ 698us) server_negotiation.cc:299] Negotiation successful
0806 02:14:09.223705 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":110,"threads_started":1}
I20260806 02:14:09.225900 30666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:09.207694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42121 (local address 127.18.70.65:48727)
0806 02:14:09.209340 (+ 1646us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:09.209399 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:09.210908 (+ 1509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:09.211846 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:09.211858 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:09.212219 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:14:09.212930 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:09.212971 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:09.214186 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:09.214189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:09.215611 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:09.215622 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:09.216381 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:09.216385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:09.216580 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:09.217240 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:14:09.217261 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:14:09.218979 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:14:09.221478 (+ 2499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:09.221486 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:09.221504 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:09.221857 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:09.222251 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:09.222256 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:09.222261 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:09.222444 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:09.222921 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:14:09.222927 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:14:09.223250 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 02:14:09.224145 (+ 895us) client_negotiation.cc:240] Negotiation successful
0806 02:14:09.224799 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1400,"thread_start_us":122,"threads_started":1}
I20260806 02:14:09.227623 30664 heartbeater.cc:347] Connected to a master server at 127.18.70.126:42121
I20260806 02:14:09.227824 30664 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:09.228430 30664 heartbeater.cc:511] Master 127.18.70.126:42121 requested a full tablet report, sending...
I20260806 02:14:09.230320 30435 ts_manager.cc:194] Registered new tserver with Master: c87d226ec5aa44778d8309721e439a95 (127.18.70.65:44495)
I20260806 02:14:09.232002 30435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:48727
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:14:09.264940 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.70.84:52927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42121
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 02:14:09 dist-test-slave-6k22 krb5kdc[29934](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785982449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 02:14:09.435098 30672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:14:09.435484 30672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:14:09.435585 30672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:14:09.441314 30672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:14:09.441408 30672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:14:09.441525 30672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:14:09.447340 30672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.70.84:52927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:14:09.448460 30672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:14:09.449839 30672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:14:09.462333 30680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:09.462347 30679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:09.462462 30682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:09.463371 30672 server_base.cc:1034] running on GCE node
I20260806 02:14:09.463824 30672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:14:09.464484 30672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:14:09.465694 30672 hybrid_clock.cc:648] HybridClock initialized: now 1785982449465537 us; error 134 us; skew 500 ppm
Aug 06 02:14:09 dist-test-slave-6k22 krb5kdc[29934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:09.468900 30672 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:14:09.470080 30672 webserver.cc:460] Webserver started at http://127.18.70.66:43791/ using document root <none> and password file <none>
I20260806 02:14:09.470870 30672 fs_manager.cc:359] Metadata directory not provided
I20260806 02:14:09.470969 30672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:14:09.471243 30672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 02:14:09 dist-test-slave-6k22 krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:09.487926 30672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "1d1ef57689be42298c1e9d5d2dffc3ed"
format_stamp: "Formatted at 2026-08-06 02:14:09 on dist-test-slave-6k22"
server_key: "4f5e4469c6ec656881146c98b8535b2e"
server_key_iv: "cdcf02a3741865f80709490f24fed959"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:09.488708 30672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "1d1ef57689be42298c1e9d5d2dffc3ed"
format_stamp: "Formatted at 2026-08-06 02:14:09 on dist-test-slave-6k22"
server_key: "4f5e4469c6ec656881146c98b8535b2e"
server_key_iv: "cdcf02a3741865f80709490f24fed959"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:09.493522 30672 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 02:14:09.513527 30690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:09.520467 30672 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.004s sys 0.000s
I20260806 02:14:09.520720 30672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "1d1ef57689be42298c1e9d5d2dffc3ed"
format_stamp: "Formatted at 2026-08-06 02:14:09 on dist-test-slave-6k22"
server_key: "4f5e4469c6ec656881146c98b8535b2e"
server_key_iv: "cdcf02a3741865f80709490f24fed959"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:09.520967 30672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:14:09.554817 30672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:14:09.559192 30672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:14:09.559458 30672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:14:09.560284 30672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:14:09.561391 30672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:14:09.561492 30672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:09.561589 30672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:14:09.561650 30672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:09.583353 30672 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:38991
I20260806 02:14:09.584671 30672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:14:09.585647 30805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:38991 every 8 connection(s)
I20260806 02:14:09.590114 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 30672
I20260806 02:14:09.590255 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:14:09.590559 18713 external_mini_cluster.cc:1428] Setting key 65746e43ecc64f42ab3e46b292797104
Aug 06 02:14:09 dist-test-slave-6k22 krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
I20260806 02:14:09.615826 30667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:09.591160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:50465 (local address 127.18.70.126:42121)
0806 02:14:09.591323 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:09.591328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:09.595396 (+ 4068us) server_negotiation.cc:407] Connection header received
0806 02:14:09.599719 (+ 4323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:09.599723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:09.599780 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:09.599837 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:09.601574 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:09.602339 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:09.604728 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:09.605079 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:09.605180 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:09.611218 (+ 6038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:09.611222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:09.611225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:09.611253 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:09.613145 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:09.613756 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:09.613760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:09.613762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:09.613853 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:09.614328 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:09.614332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:09.614334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:09.614588 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:09.614753 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:09.615463 (+ 710us) server_negotiation.cc:299] Negotiation successful
0806 02:14:09.615633 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 02:14:09.617805 30809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:09.588916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42121 (local address 127.18.70.66:50465)
0806 02:14:09.595236 (+ 6320us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:09.595297 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:09.596406 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:09.600290 (+ 3884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:09.600299 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:09.600679 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:14:09.601411 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:09.601432 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:09.602565 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:09.602568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:09.604009 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:09.604020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:09.605386 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:09.605390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:09.605585 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:09.609065 (+ 3480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:14:09.609086 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:14:09.610992 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:14:09.613270 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:09.613277 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:09.613295 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:09.613624 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:09.613954 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:09.613958 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:09.613961 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:09.614213 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:09.614807 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:14:09.614814 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:14:09.615152 (+ 338us) client_negotiation.cc:769] Sending connection context
0806 02:14:09.616299 (+ 1147us) client_negotiation.cc:240] Negotiation successful
0806 02:14:09.616688 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6073,"thread_start_us":140,"threads_started":1}
I20260806 02:14:09.619407 30806 heartbeater.cc:347] Connected to a master server at 127.18.70.126:42121
I20260806 02:14:09.619697 30806 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:09.620306 30806 heartbeater.cc:511] Master 127.18.70.126:42121 requested a full tablet report, sending...
I20260806 02:14:09.621522 30435 ts_manager.cc:194] Registered new tserver with Master: 1d1ef57689be42298c1e9d5d2dffc3ed (127.18.70.66:38991)
I20260806 02:14:09.622280 30435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:50465
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:14:09.651894 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.70.84:52927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42121
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:14:09.837782 30812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:14:09.838220 30812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:14:09.838340 30812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:14:09.844637 30812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:14:09.844803 30812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:14:09.844964 30812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:14:09.852905 30812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.70.84:52927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:14:09.855163 30812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:14:09.856598 30812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:14:09.870793 30818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:09.871359 30817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:09.871671 30820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:09.873081 30812 server_base.cc:1034] running on GCE node
I20260806 02:14:09.873903 30812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:14:09.874667 30812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 06 02:14:09 dist-test-slave-6k22 krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:09.891489 30812 hybrid_clock.cc:648] HybridClock initialized: now 1785982449891427 us; error 77 us; skew 500 ppm
Aug 06 02:14:09 dist-test-slave-6k22 krb5kdc[29934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:09.896162 30812 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:14:09.897311 30812 webserver.cc:460] Webserver started at http://127.18.70.67:34565/ using document root <none> and password file <none>
I20260806 02:14:09.898065 30812 fs_manager.cc:359] Metadata directory not provided
I20260806 02:14:09.898202 30812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:14:09.898527 30812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 02:14:09 dist-test-slave-6k22 krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:09.928565 30812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "4de7439c4c364764aae96fd3fb7db758"
format_stamp: "Formatted at 2026-08-06 02:14:09 on dist-test-slave-6k22"
server_key: "dcd9a75aecc911cf26c13048af47201e"
server_key_iv: "f80b39ed8329206fd840b7fc8eeff181"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:09.929333 30812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "4de7439c4c364764aae96fd3fb7db758"
format_stamp: "Formatted at 2026-08-06 02:14:09 on dist-test-slave-6k22"
server_key: "dcd9a75aecc911cf26c13048af47201e"
server_key_iv: "f80b39ed8329206fd840b7fc8eeff181"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:09.939286 30812 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.000s
I20260806 02:14:09.976009 30827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:09.978297 30812 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260806 02:14:09.978477 30812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "4de7439c4c364764aae96fd3fb7db758"
format_stamp: "Formatted at 2026-08-06 02:14:09 on dist-test-slave-6k22"
server_key: "dcd9a75aecc911cf26c13048af47201e"
server_key_iv: "f80b39ed8329206fd840b7fc8eeff181"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:09.978643 30812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:14:10.002951 30812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:14:10.006901 30812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:14:10.007159 30812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:14:10.007937 30812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:14:10.009131 30812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:14:10.009264 30812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:10.009372 30812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:14:10.009443 30812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:10.027096 30812 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:34929
I20260806 02:14:10.028467 30812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:14:10.033138 30947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:34929 every 8 connection(s)
I20260806 02:14:10.037313 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 30812
I20260806 02:14:10.037464 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:14:10.037787 18713 external_mini_cluster.cc:1428] Setting key f6f38d70c6e33be50ceb1a62856d0a34
Aug 06 02:14:10 dist-test-slave-6k22 krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:14:10.055545 30667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.038486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:43311 (local address 127.18.70.126:42121)
0806 02:14:10.038640 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:10.038646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:10.040604 (+ 1958us) server_negotiation.cc:407] Connection header received
0806 02:14:10.042232 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:10.042236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:10.042293 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:10.042350 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:10.044229 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.045167 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.046980 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.047352 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.047437 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.050823 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:10.050827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:10.050830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:10.050862 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:10.052823 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:10.053452 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:10.053457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:10.053459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:10.053570 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:10.054067 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:10.054071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:10.054073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:10.054335 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:10.054435 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:10.055211 (+ 776us) server_negotiation.cc:299] Negotiation successful
0806 02:14:10.055353 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:14:10.057025 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.037197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42121 (local address 127.18.70.67:43311)
0806 02:14:10.038864 (+ 1667us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:10.040498 (+ 1634us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:10.041958 (+ 1460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:10.042811 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:10.042823 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:10.043235 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:14:10.044027 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:10.044048 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.045335 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.045338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:10.046830 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:10.046842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.047672 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.047676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:10.047878 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.048621 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:14:10.048646 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:14:10.050587 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:14:10.052948 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:10.052952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:10.052965 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:10.053230 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:10.053712 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:10.053716 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:10.053718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:10.053928 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:10.054516 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:14:10.054524 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:14:10.054872 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 02:14:10.055862 (+ 990us) client_negotiation.cc:240] Negotiation successful
0806 02:14:10.056176 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1252,"thread_start_us":153,"threads_started":1}
I20260806 02:14:10.058231 30948 heartbeater.cc:347] Connected to a master server at 127.18.70.126:42121
I20260806 02:14:10.058512 30948 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:10.059161 30948 heartbeater.cc:511] Master 127.18.70.126:42121 requested a full tablet report, sending...
I20260806 02:14:10.060441 30435 ts_manager.cc:194] Registered new tserver with Master: 4de7439c4c364764aae96fd3fb7db758 (127.18.70.67:34929)
I20260806 02:14:10.061185 30435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:43311
I20260806 02:14:10.064491 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:14:10.075781 30667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.067282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.18.70.126:42121)
0806 02:14:10.067402 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:10.067408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:10.067493 (+ 85us) server_negotiation.cc:407] Connection header received
0806 02:14:10.067668 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:10.067672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:10.067718 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:10.067820 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:10.068570 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.069139 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.070036 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.070346 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.070476 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.072276 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:10.072280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:10.072283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:10.072310 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:10.073858 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:10.074378 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:10.074384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:10.074386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:10.074477 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:10.074788 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:10.074793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:10.074795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:10.074988 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:10.075137 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:10.075419 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 02:14:10.075535 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:14:10.079008 30435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:14:10.082376 30435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:14:10.107788 30960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.095322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:34929 (local address 127.0.0.1:53326)
0806 02:14:10.098795 (+ 3473us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:10.098809 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:10.098896 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:10.102446 (+ 3550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:10.102449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:10.102480 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:14:10.102564 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:10.102570 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.103893 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.103896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:10.105029 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:10.105041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.107128 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.107133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:10.107256 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.107303 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 02:14:10.107427 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 02:14:10.107603 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3411,"thread_start_us":91,"threads_started":1}
I20260806 02:14:10.108496 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.095818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53326 (local address 127.18.70.67:34929)
0806 02:14:10.097073 (+ 1255us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:10.097086 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:10.098997 (+ 1911us) server_negotiation.cc:407] Connection header received
0806 02:14:10.099090 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:10.099095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:10.099290 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:10.099451 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:14:10.102707 (+ 3256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.103759 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.105204 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.105989 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.106099 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.106259 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:10.108197 (+ 1938us) server_negotiation.cc:299] Negotiation successful
0806 02:14:10.108342 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1140,"thread_start_us":145,"threads_started":1}
I20260806 02:14:10.113509 30878 tablet_service.cc:1467] Processing CreateTablet for tablet 9821cf0a688844a8835994f209f8c095 (DEFAULT_TABLE table=test-table [id=c48095e048cc44b88f4c10f26c2db152]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:14:10.114639 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.092982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:38991 (local address 127.0.0.1:50290)
0806 02:14:10.096530 (+ 3548us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:10.096581 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:10.096732 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:10.106431 (+ 9699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:10.106435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:10.106451 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:14:10.106532 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:10.106538 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.110226 (+ 3688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.110229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:10.111464 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:10.111475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.114271 (+ 2796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.114276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:10.114372 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.114395 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 02:14:10.114443 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 02:14:10.114520 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3366,"thread_start_us":101,"threads_started":1}
I20260806 02:14:10.115401 30878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9821cf0a688844a8835994f209f8c095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:14:10.115425 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.093109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50290 (local address 127.18.70.66:38991)
0806 02:14:10.105817 (+ 12708us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:10.105834 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:10.105858 (+ 24us) server_negotiation.cc:407] Connection header received
0806 02:14:10.105931 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:10.105936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:10.106109 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:10.106282 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:14:10.106677 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.107840 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.111663 (+ 3823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.112484 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.114934 (+ 2450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.115086 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:10.115169 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 02:14:10.115273 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12596,"thread_start_us":99,"threads_started":1}
I20260806 02:14:10.118327 30737 tablet_service.cc:1467] Processing CreateTablet for tablet 9821cf0a688844a8835994f209f8c095 (DEFAULT_TABLE table=test-table [id=c48095e048cc44b88f4c10f26c2db152]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:14:10.119381 30737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9821cf0a688844a8835994f209f8c095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:14:10.109935 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.094211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:44495 (local address 127.0.0.1:53428)
0806 02:14:10.097221 (+ 3010us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:10.097235 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:10.097329 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:10.098380 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:10.098388 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:10.098413 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:14:10.098555 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:10.098563 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.100896 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.100900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:10.102302 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:10.102314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.109420 (+ 7106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.109425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:10.109520 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.109545 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 02:14:10.109673 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 02:14:10.109792 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2947,"thread_start_us":101,"threads_started":1}
I20260806 02:14:10.126304 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.094518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.18.70.65:44495)
0806 02:14:10.097596 (+ 3078us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:10.097603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:10.097642 (+ 39us) server_negotiation.cc:407] Connection header received
0806 02:14:10.097730 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:10.097736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:10.097989 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:10.098159 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:14:10.099653 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.100752 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.106324 (+ 5572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.109075 (+ 2751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.109200 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.109352 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:10.126006 (+ 16654us) server_negotiation.cc:299] Negotiation successful
0806 02:14:10.126131 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2955,"thread_start_us":110,"threads_started":1}
I20260806 02:14:10.129602 30594 tablet_service.cc:1467] Processing CreateTablet for tablet 9821cf0a688844a8835994f209f8c095 (DEFAULT_TABLE table=test-table [id=c48095e048cc44b88f4c10f26c2db152]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:14:10.130693 30594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9821cf0a688844a8835994f209f8c095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:14:10.136518 30968 tablet_bootstrap.cc:492] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed: Bootstrap starting.
I20260806 02:14:10.138952 30966 tablet_bootstrap.cc:492] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758: Bootstrap starting.
I20260806 02:14:10.139036 30968 tablet_bootstrap.cc:654] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed: Neither blocks nor log segments found. Creating new log.
I20260806 02:14:10.140059 30968 log.cc:824] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed: Log is configured to *not* fsync() on all Append() calls
I20260806 02:14:10.140458 30969 tablet_bootstrap.cc:492] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95: Bootstrap starting.
I20260806 02:14:10.142555 30969 tablet_bootstrap.cc:654] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95: Neither blocks nor log segments found. Creating new log.
I20260806 02:14:10.143215 30966 tablet_bootstrap.cc:654] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758: Neither blocks nor log segments found. Creating new log.
I20260806 02:14:10.143489 30969 log.cc:824] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95: Log is configured to *not* fsync() on all Append() calls
I20260806 02:14:10.144124 30966 log.cc:824] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758: Log is configured to *not* fsync() on all Append() calls
I20260806 02:14:10.145601 30969 tablet_bootstrap.cc:492] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95: No bootstrap required, opened a new log
I20260806 02:14:10.145807 30969 ts_tablet_manager.cc:1397] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 02:14:10.146402 30968 tablet_bootstrap.cc:492] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed: No bootstrap required, opened a new log
I20260806 02:14:10.146601 30968 ts_tablet_manager.cc:1397] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260806 02:14:10.146765 30966 tablet_bootstrap.cc:492] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758: No bootstrap required, opened a new log
I20260806 02:14:10.146958 30966 ts_tablet_manager.cc:1397] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 02:14:10.150525 30969 raft_consensus.cc:348] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1ef57689be42298c1e9d5d2dffc3ed" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38991 } } peers { permanent_uuid: "c87d226ec5aa44778d8309721e439a95" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 44495 } } peers { permanent_uuid: "4de7439c4c364764aae96fd3fb7db758" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34929 } }
I20260806 02:14:10.150779 30969 raft_consensus.cc:374] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:14:10.150826 30969 raft_consensus.cc:729] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87d226ec5aa44778d8309721e439a95, State: Initialized, Role: FOLLOWER
I20260806 02:14:10.151054 30968 raft_consensus.cc:348] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1ef57689be42298c1e9d5d2dffc3ed" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38991 } } peers { permanent_uuid: "c87d226ec5aa44778d8309721e439a95" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 44495 } } peers { permanent_uuid: "4de7439c4c364764aae96fd3fb7db758" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34929 } }
I20260806 02:14:10.151054 30966 raft_consensus.cc:348] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1ef57689be42298c1e9d5d2dffc3ed" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38991 } } peers { permanent_uuid: "c87d226ec5aa44778d8309721e439a95" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 44495 } } peers { permanent_uuid: "4de7439c4c364764aae96fd3fb7db758" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34929 } }
I20260806 02:14:10.151330 30968 raft_consensus.cc:374] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:14:10.151330 30966 raft_consensus.cc:374] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:14:10.151415 30966 raft_consensus.cc:729] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de7439c4c364764aae96fd3fb7db758, State: Initialized, Role: FOLLOWER
I20260806 02:14:10.151415 30968 raft_consensus.cc:729] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1ef57689be42298c1e9d5d2dffc3ed, State: Initialized, Role: FOLLOWER
I20260806 02:14:10.151858 30969 consensus_queue.cc:260] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1ef57689be42298c1e9d5d2dffc3ed" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38991 } } peers { permanent_uuid: "c87d226ec5aa44778d8309721e439a95" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 44495 } } peers { permanent_uuid: "4de7439c4c364764aae96fd3fb7db758" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34929 } }
I20260806 02:14:10.152230 30966 consensus_queue.cc:260] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1ef57689be42298c1e9d5d2dffc3ed" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38991 } } peers { permanent_uuid: "c87d226ec5aa44778d8309721e439a95" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 44495 } } peers { permanent_uuid: "4de7439c4c364764aae96fd3fb7db758" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34929 } }
I20260806 02:14:10.153033 30969 ts_tablet_manager.cc:1428] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 02:14:10.153280 30966 ts_tablet_manager.cc:1428] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 02:14:10.153429 30664 heartbeater.cc:503] Master 127.18.70.126:42121 was elected leader, sending a full tablet report...
I20260806 02:14:10.153987 30968 consensus_queue.cc:260] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1ef57689be42298c1e9d5d2dffc3ed" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38991 } } peers { permanent_uuid: "c87d226ec5aa44778d8309721e439a95" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 44495 } } peers { permanent_uuid: "4de7439c4c364764aae96fd3fb7db758" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34929 } }
I20260806 02:14:10.154443 30948 heartbeater.cc:503] Master 127.18.70.126:42121 was elected leader, sending a full tablet report...
I20260806 02:14:10.155053 30968 ts_tablet_manager.cc:1428] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 02:14:10.155376 30806 heartbeater.cc:503] Master 127.18.70.126:42121 was elected leader, sending a full tablet report...
I20260806 02:14:10.206346 30973 raft_consensus.cc:482] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260806 02:14:10.206431 30665 tablet.cc:2369] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:14:10.207190 30973 raft_consensus.cc:504] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1ef57689be42298c1e9d5d2dffc3ed" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38991 } } peers { permanent_uuid: "c87d226ec5aa44778d8309721e439a95" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 44495 } } peers { permanent_uuid: "4de7439c4c364764aae96fd3fb7db758" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34929 } }
I20260806 02:14:10.210170 30973 leader_election.cc:288] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d1ef57689be42298c1e9d5d2dffc3ed (127.18.70.66:38991), 4de7439c4c364764aae96fd3fb7db758 (127.18.70.67:34929)
I20260806 02:14:10.215126 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.209258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:52845 (local address 127.18.70.66:38991)
0806 02:14:10.209661 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:10.209667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:10.210609 (+ 942us) server_negotiation.cc:407] Connection header received
0806 02:14:10.210823 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:10.210826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:10.210880 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:10.210929 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:14:10.211403 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.212223 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.213853 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.214465 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.214573 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.214625 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:10.214939 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 02:14:10.215011 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283}
I20260806 02:14:10.215684 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.208795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:38991 (local address 127.18.70.65:52845)
0806 02:14:10.210517 (+ 1722us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:10.210532 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:10.210686 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:10.211089 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:10.211092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:10.211109 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:14:10.211263 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:10.211269 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.212397 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.212403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:10.213710 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:10.213721 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.214721 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.214724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:10.214814 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.214835 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 02:14:10.215473 (+ 638us) client_negotiation.cc:240] Negotiation successful
0806 02:14:10.215549 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1652,"thread_start_us":117,"threads_started":1}
I20260806 02:14:10.216506 30742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9821cf0a688844a8835994f209f8c095" candidate_uuid: "c87d226ec5aa44778d8309721e439a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1ef57689be42298c1e9d5d2dffc3ed" is_pre_election: true
I20260806 02:14:10.216805 30742 raft_consensus.cc:2454] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c87d226ec5aa44778d8309721e439a95 in term 0.
I20260806 02:14:10.217319 30542 leader_election.cc:302] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1ef57689be42298c1e9d5d2dffc3ed, c87d226ec5aa44778d8309721e439a95; no voters:
I20260806 02:14:10.217649 30973 raft_consensus.cc:2781] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:14:10.217751 30973 raft_consensus.cc:482] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:14:10.217821 30973 raft_consensus.cc:3037] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:14:10.220063 30973 raft_consensus.cc:504] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1ef57689be42298c1e9d5d2dffc3ed" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38991 } } peers { permanent_uuid: "c87d226ec5aa44778d8309721e439a95" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 44495 } } peers { permanent_uuid: "4de7439c4c364764aae96fd3fb7db758" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34929 } }
I20260806 02:14:10.220628 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.210007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:55599 (local address 127.18.70.67:34929)
0806 02:14:10.210125 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:10.210130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:10.215593 (+ 5463us) server_negotiation.cc:407] Connection header received
0806 02:14:10.215825 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:10.215830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:10.215889 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:10.215943 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:14:10.216388 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.217091 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.218784 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.219318 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.219403 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.219447 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:10.220438 (+ 991us) server_negotiation.cc:299] Negotiation successful
0806 02:14:10.220516 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:14:10.220912 30973 leader_election.cc:288] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [CANDIDATE]: Term 1 election: Requested vote from peers 1d1ef57689be42298c1e9d5d2dffc3ed (127.18.70.66:38991), 4de7439c4c364764aae96fd3fb7db758 (127.18.70.67:34929)
I20260806 02:14:10.221199 30977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.209609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:34929 (local address 127.18.70.65:55599)
0806 02:14:10.215452 (+ 5843us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:10.215472 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:10.215676 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:10.216104 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:10.216122 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:10.216142 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:14:10.216239 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:10.216246 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.217356 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.217360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:10.218649 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:10.218660 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.220130 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.220133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:10.220302 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.220325 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 02:14:10.220926 (+ 601us) client_negotiation.cc:240] Negotiation successful
0806 02:14:10.221057 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5774,"thread_start_us":101,"threads_started":1}
I20260806 02:14:10.221172 30742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9821cf0a688844a8835994f209f8c095" candidate_uuid: "c87d226ec5aa44778d8309721e439a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1ef57689be42298c1e9d5d2dffc3ed"
I20260806 02:14:10.221311 30742 raft_consensus.cc:3037] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:14:10.222005 30899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9821cf0a688844a8835994f209f8c095" candidate_uuid: "c87d226ec5aa44778d8309721e439a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de7439c4c364764aae96fd3fb7db758" is_pre_election: true
I20260806 02:14:10.222288 30899 raft_consensus.cc:2454] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c87d226ec5aa44778d8309721e439a95 in term 0.
I20260806 02:14:10.222579 30898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9821cf0a688844a8835994f209f8c095" candidate_uuid: "c87d226ec5aa44778d8309721e439a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de7439c4c364764aae96fd3fb7db758"
I20260806 02:14:10.222676 30898 raft_consensus.cc:3037] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:14:10.222724 30742 raft_consensus.cc:2454] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c87d226ec5aa44778d8309721e439a95 in term 1.
I20260806 02:14:10.223089 30542 leader_election.cc:302] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d1ef57689be42298c1e9d5d2dffc3ed, c87d226ec5aa44778d8309721e439a95; no voters:
I20260806 02:14:10.223290 30973 raft_consensus.cc:2781] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:14:10.223512 30973 raft_consensus.cc:686] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [term 1 LEADER]: Becoming Leader. State: Replica: c87d226ec5aa44778d8309721e439a95, State: Running, Role: LEADER
I20260806 02:14:10.223830 30898 raft_consensus.cc:2454] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c87d226ec5aa44778d8309721e439a95 in term 1.
I20260806 02:14:10.223918 30973 consensus_queue.cc:237] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1ef57689be42298c1e9d5d2dffc3ed" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38991 } } peers { permanent_uuid: "c87d226ec5aa44778d8309721e439a95" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 44495 } } peers { permanent_uuid: "4de7439c4c364764aae96fd3fb7db758" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34929 } }
I20260806 02:14:10.227800 30434 catalog_manager.cc:5495] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 reported cstate change: term changed from 0 to 1, leader changed from <none> to c87d226ec5aa44778d8309721e439a95 (127.18.70.65). New cstate: current_term: 1 leader_uuid: "c87d226ec5aa44778d8309721e439a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1ef57689be42298c1e9d5d2dffc3ed" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c87d226ec5aa44778d8309721e439a95" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 44495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4de7439c4c364764aae96fd3fb7db758" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34929 } health_report { overall_health: UNKNOWN } } }
I20260806 02:14:10.260164 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.254718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53434 (local address 127.18.70.65:44495)
0806 02:14:10.254857 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:10.254865 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:10.254883 (+ 18us) server_negotiation.cc:407] Connection header received
0806 02:14:10.255155 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:10.255159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:10.255263 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:10.255388 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:14:10.255864 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.256811 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.258079 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:10.258417 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:10.258529 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:10.259077 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:14:10.259693 (+ 616us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:14:10.259860 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:10.259903 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 02:14:10.259964 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 02:14:10.269599 30898 raft_consensus.cc:1261] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758 [term 1 FOLLOWER]: Refusing update from remote peer c87d226ec5aa44778d8309721e439a95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:14:10.270747 30973 consensus_queue.cc:1035] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4de7439c4c364764aae96fd3fb7db758" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 34929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:14:10.271639 30742 raft_consensus.cc:1261] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed [term 1 FOLLOWER]: Refusing update from remote peer c87d226ec5aa44778d8309721e439a95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:14:10.272522 30978 consensus_queue.cc:1035] T 9821cf0a688844a8835994f209f8c095 P c87d226ec5aa44778d8309721e439a95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1ef57689be42298c1e9d5d2dffc3ed" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:14:10.283411 30981 mvcc.cc:205] Tried to move back new op lower bound from 7315384116291735552 to 7315384116120956928. Current Snapshot: MvccSnapshot[applied={T|T < 7315384116291735552}]
I20260806 02:14:10.283900 30982 mvcc.cc:205] Tried to move back new op lower bound from 7315384116291735552 to 7315384116120956928. Current Snapshot: MvccSnapshot[applied={T|T < 7315384116291735552}]
I20260806 02:14:10.295413 30984 mvcc.cc:205] Tried to move back new op lower bound from 7315384116291735552 to 7315384116120956928. Current Snapshot: MvccSnapshot[applied={T|T < 7315384116291735552}]
I20260806 02:14:10.296610 30435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:14:10.296792 30435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:14:10.299540 30435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce560bed780f4689b4fc249eeffe34e7: Sending DeleteTablet for 3 replicas of tablet 9821cf0a688844a8835994f209f8c095
I20260806 02:14:10.300756 30878 tablet_service.cc:1514] Processing DeleteTablet for tablet 9821cf0a688844a8835994f209f8c095 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:14:10 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:53326
I20260806 02:14:10.301088 30594 tablet_service.cc:1514] Processing DeleteTablet for tablet 9821cf0a688844a8835994f209f8c095 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:14:10 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:53428
I20260806 02:14:10.301455 30992 tablet_replica.cc:330] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758: stopping tablet replica
I20260806 02:14:10.301717 30737 tablet_service.cc:1514] Processing DeleteTablet for tablet 9821cf0a688844a8835994f209f8c095 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:14:10 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:50290
I20260806 02:14:10.302771 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 30522
I20260806 02:14:10.307282 30992 raft_consensus.cc:2229] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:14:10.307758 30992 raft_consensus.cc:2258] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:14:10.313884 30992 ts_tablet_manager.cc:1905] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 02:14:10.317654 30422 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:44495 (error 108)
I20260806 02:14:10.317745 30994 tablet_replica.cc:330] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed: stopping tablet replica
I20260806 02:14:10.318140 30994 raft_consensus.cc:2229] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:14:10.318563 30994 raft_consensus.cc:2258] T 9821cf0a688844a8835994f209f8c095 P 1d1ef57689be42298c1e9d5d2dffc3ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:14:10.318960 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 30672
I20260806 02:14:10.321245 30992 ts_tablet_manager.cc:1918] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:14:10.321753 30992 log.cc:1196] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/9821cf0a688844a8835994f209f8c095
I20260806 02:14:10.322235 30992 ts_tablet_manager.cc:1939] T 9821cf0a688844a8835994f209f8c095 P 4de7439c4c364764aae96fd3fb7db758: Deleting consensus metadata
I20260806 02:14:10.324304 30419 catalog_manager.cc:4841] TS 4de7439c4c364764aae96fd3fb7db758 (127.18.70.67:34929): tablet 9821cf0a688844a8835994f209f8c095 (table test-table [id=c48095e048cc44b88f4c10f26c2db152]) successfully deleted
Aug 06 02:14:10 dist-test-slave-6k22 krb5kdc[29934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:10.330065 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 30812
I20260806 02:14:10.331267 30960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.330376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:44495 (local address 127.0.0.1:53448)
0806 02:14:10.331039 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:10.331144 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:44495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525}
I20260806 02:14:10.331578 30960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:10.330775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:38991 (local address 127.0.0.1:50304)
0806 02:14:10.331444 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:10.331498 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.66:38991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":598}
W20260806 02:14:10.334184 30422 catalog_manager.cc:4584] TS c87d226ec5aa44778d8309721e439a95 (127.18.70.65:44495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9821cf0a688844a8835994f209f8c095: Network error: Client connection negotiation failed: client connection to 127.18.70.65:44495: connect: Connection refused (error 111)
I20260806 02:14:10.342389 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 30403
I20260806 02:14:10.351351 18713 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 02:14:10.697515 18713 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 02:14:10.697682 18713 mini_ranger.cc:66] Stopping Ranger...
I20260806 02:14:11.048535 18713 mini_ranger.cc:68] Stopped Ranger
2026-08-06 02:14:11.048 UTC [29966] LOG: received smart shutdown request
2026-08-06 02:14:11.052 UTC [29966] LOG: background worker "logical replication launcher" (PID 29976) exited with exit code 1
2026-08-06 02:14:11.052 UTC [29970] LOG: shutting down
2026-08-06 02:14:11.204 UTC [29966] LOG: database system is shut down
2026-08-06T02:14:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47072 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:14:11 dist-test-slave-6k22 krb5kdc[31005](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:14:11 dist-test-slave-6k22 krb5kdc[31005](info): set up 2 sockets
Aug 06 02:14:11 dist-test-slave-6k22 krb5kdc[31005](info): commencing operation
krb5kdc: starting...
W20260806 02:14:12.416504 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:14:12 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:14:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:14:12Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.70.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.70.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/rangeradmin_127.18.70.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/rangeradmin_127.18.70.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.70.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.70.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/rangerlookup_127.18.70.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/rangerlookup_127.18.70.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.70.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.70.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/HTTP_127.18.70.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/HTTP_127.18.70.84@KRBTEST.COM.keytab.
I20260806 02:14:12.582298 18713 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskVlzDnt/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/postgres -l logfile start
2026-08-06 02:14:14.967 UTC [31037] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 02:14:14.967 UTC [31037] LOG: listening on IPv4 address "127.18.70.84", port 59709
2026-08-06 02:14:14.970 UTC [31037] LOG: listening on Unix socket "/tmp/.s.PGSQL.59709"
2026-08-06 02:14:14.991 UTC [31041] LOG: database system was shut down at 2026-08-06 02:14:13 UTC
2026-08-06 02:14:14.995 UTC [31037] LOG: database system is ready to accept connections
I20260806 02:14:15.981026 18713 mini_postgres.cc:96] Postgres bound to 59709
2026-08-06 02:14:15.988 UTC [31050] FATAL: database "slave" does not exist
127.18.70.84:59709 - accepting connections
I20260806 02:14:15.988837 18713 mini_ranger.cc:161] Starting Ranger...
I20260806 02:14:16.004158 18713 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 02:14:16.455227 18713 mini_ranger.cc:87] Created ranger Postgres database
I20260806 02:14:16.455304 18713 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-admin
2026-08-06 02:14:16,756 [I] DB FLAVOR :POSTGRES
2026-08-06 02:14:16,756 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 02:14:16,756 [I] Checking connection..
2026-08-06 02:14:16,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 02:14:17,025 [I] Checking connection passed.
2026-08-06 02:14:17,025 [I] --------- Verifying version history table ---------
2026-08-06 02:14:17,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 02:14:17,290 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 02:14:17,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 02:14:17,554 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 02:14:17,555 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 02:14:17,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 02:14:17.796 UTC [31114] WARNING: there is no transaction in progress
2026-08-06 02:14:17,803 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 02:14:17,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 02:14:18,041 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 02:14:18,041 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 02:14:18,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 02:14:18,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 02:14:18,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6k22.c.gcp-upstream.internal','N') ;"
2026-08-06 02:14:18,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 02:14:19,049 [I] Table x_portal_user does not exist in database ranger
2026-08-06 02:14:19,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 02:14:19,340 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 02:14:19,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 02:14:19,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 02:14:19,859 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 02:14:19,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 02:14:20.784 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.799 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.814 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.828 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.845 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.865 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.921 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.928 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.941 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.950 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.959 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.972 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.981 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.987 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:20.993 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:21.000 UTC [31227] WARNING: there is no transaction in progress
2026-08-06 02:14:21,621 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 02:14:21,622 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 02:14:21,870 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 02:14:21,870 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 02:14:22,125 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 02:14:22,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 02:14:22,393 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 02:14:22,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 02:14:22,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 02:14:22,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6k22.c.gcp-upstream.internal';"
2026-08-06 02:14:23,162 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 02:14:23,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 02:14:23,392 [I] DB_PATCHES have already been applied
I20260806 02:14:23.398747 18713 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-admin:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskVlzDnt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 02:14:23.398850 18713 mini_ranger.cc:193] Using host: 127.18.70.84
I20260806 02:14:23.403249 18713 mini_ranger.cc:227] Ranger admin URL: http://127.18.70.84:38401
Aug 06, 2026 2:14:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 2:14:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 2:14:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 2:14:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 2:14:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 2:14:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 2:14:24 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 02:14:24.530642 18713 mini_ranger.cc:160] Time spent starting Ranger: real 8.542s user 0.000s sys 0.011s
Aug 06, 2026 2:14:43 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 2:14:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 2:14:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 2:14:43 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 2:14:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 2:14:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 2:14:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 2:14:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 2:14:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 2:14:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 2:14:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 2:14:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 2:14:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 2:14:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 2:14:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 2:14:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 2:14:43 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 2:14:44 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 02:14:44.792758 18713 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.70.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.70.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/rangerkms_127.18.70.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.70.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/rangerkms_127.18.70.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.70.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.70.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.70.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/HTTP_127.18.70.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.70.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/HTTP_127.18.70.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 02:14:44 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982484, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 02:14:44.879096 18713 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 02:14:44.888962 18713 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 02:14:45.716725 18713 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 02:14:45.716823 18713 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-kms
I20260806 02:14:45.716845 18713 mini_ranger_kms.cc:227] Using postgres at 127.18.70.84:59709
2026-08-06 02:14:45,899 [I] DB FLAVOR :POSTGRES
2026-08-06 02:14:45,899 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 02:14:45,899 [I] Checking connection
2026-08-06 02:14:45,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 02:14:46,138 [I] connection success
2026-08-06 02:14:46,138 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 02:14:46,138 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 02:14:46,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 02:14:46,386 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 02:14:46,386 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 02:14:46,386 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 02:14:46,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.70.84:59709/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 02:14:46,638 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 02:14:47.113189 18713 mini_ranger_kms.cc:310] Created kms service
I20260806 02:14:47.291730 18713 mini_ranger_kms.cc:326] Created kudu user
I20260806 02:14:47.335661 18713 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 02:14:47 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982484, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:47.739058 18713 mini_ranger_kms.cc:384] Added ranger policy
I20260806 02:14:47.739233 18713 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-kms:/tmp/dist-test-taskVlzDnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskVlzDnt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 02:14:47.739257 18713 mini_ranger_kms.cc:240] Using host: 127.18.70.84
I20260806 02:14:47.744367 18713 mini_ranger_kms.cc:276] Ranger KMS PID: 31415
I20260806 02:14:47.744452 18713 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.70.84:51435
Aug 06, 2026 2:14:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 2:14:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 2:14:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 2:14:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 2:14:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 2:14:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 02:14:49 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785982489, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.70.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:50.049188 18713 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.170s user 0.024s sys 0.143s
I20260806 02:14:50.049333 18713 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 02:14:50.049425 18713 mini_ranger_kms.cc:401] 127.18.70.84:51435/kms/v1/keys
Aug 06 02:14:50 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982489, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.70.84@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
Aug 06 02:14:50 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982489, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.70.84@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
Aug 06 02:14:53 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:14:53.295367 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:33443
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:33443
--ranger_config_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.70.84:51435/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:14:53.426146 31467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:14:53.426395 31467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:14:53.426433 31467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:14:53.430348 31467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:14:53.430400 31467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:14:53.430418 31467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:14:53.430433 31467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:14:53.430449 31467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:14:53.435671 31467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.70.84:51435/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:33443
--ranger_config_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:33443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:14:53.436601 31467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:14:53.437517 31467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:14:53.445748 31473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:53.445770 31476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:53.445786 31474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:53.446041 31467 server_base.cc:1034] running on GCE node
I20260806 02:14:53.446564 31467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:14:53.447688 31467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:14:53.448866 31467 hybrid_clock.cc:648] HybridClock initialized: now 1785982493448847 us; error 31 us; skew 500 ppm
Aug 06 02:14:53 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982493, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:53.456141 31467 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:14:53.457018 31467 webserver.cc:460] Webserver started at http://127.18.70.126:41333/ using document root <none> and password file <none>
I20260806 02:14:53.457567 31467 fs_manager.cc:359] Metadata directory not provided
I20260806 02:14:53.457643 31467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:14:53.457878 31467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 02:14:53 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982493, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:53.506877 31467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "ce1d7fc65f9146b5907a4cdee61776c4"
format_stamp: "Formatted at 2026-08-06 02:14:53 on dist-test-slave-6k22"
server_key: "81f22867b3b56063c4227a70cf27eac2"
server_key_iv: "a1a5c826e15f80b39494c31dc1095f6b"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:53.507613 31467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "ce1d7fc65f9146b5907a4cdee61776c4"
format_stamp: "Formatted at 2026-08-06 02:14:53 on dist-test-slave-6k22"
server_key: "81f22867b3b56063c4227a70cf27eac2"
server_key_iv: "a1a5c826e15f80b39494c31dc1095f6b"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:53.511736 31467 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 02:14:53.528493 31484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:53.530179 31467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:14:53.530359 31467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "ce1d7fc65f9146b5907a4cdee61776c4"
format_stamp: "Formatted at 2026-08-06 02:14:53 on dist-test-slave-6k22"
server_key: "81f22867b3b56063c4227a70cf27eac2"
server_key_iv: "a1a5c826e15f80b39494c31dc1095f6b"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:53.530507 31467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:14:53.559109 31467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:14:53.561654 31467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:14:53.561810 31467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:14:53.569314 31467 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:33443
I20260806 02:14:53.569348 31538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:33443 every 8 connection(s)
I20260806 02:14:53.570297 31467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:14:53.573118 31539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:14:53.573810 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 31467
I20260806 02:14:53.573943 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:14:53.574235 18713 external_mini_cluster.cc:1428] Setting key abd8024d999f4a49ee08505ae50dc0e8
I20260806 02:14:53.580044 31539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4: Bootstrap starting.
I20260806 02:14:53.582592 31539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4: Neither blocks nor log segments found. Creating new log.
I20260806 02:14:53.583364 31539 log.cc:824] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4: Log is configured to *not* fsync() on all Append() calls
Aug 06 02:14:53 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:14:53.585858 31539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4: No bootstrap required, opened a new log
I20260806 02:14:53.589979 31539 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33443 } }
I20260806 02:14:53.590232 31539 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:14:53.590299 31539 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1d7fc65f9146b5907a4cdee61776c4, State: Initialized, Role: FOLLOWER
I20260806 02:14:53.590333 31543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:53.575710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.18.70.126:33443)
0806 02:14:53.576147 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:53.576178 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:53.576224 (+ 46us) server_negotiation.cc:407] Connection header received
0806 02:14:53.577046 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:53.577070 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:53.577483 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:53.577857 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:53.578552 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:53.580045 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:53.580940 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:53.581627 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:53.581796 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:53.584585 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:53.584593 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:53.584610 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:53.584638 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:53.587326 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:53.587820 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:53.587827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:53.587833 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:53.587962 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:53.588256 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:53.588259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:53.588261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:53.588748 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:53.588959 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:53.589159 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 02:14:53.589437 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":115,"threads_started":1}
I20260806 02:14:53.590824 31539 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33443 } }
I20260806 02:14:53.590998 31539 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:14:53.591100 31539 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:14:53.591204 31539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:14:53.592330 31539 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33443 } }
I20260806 02:14:53.592733 31539 leader_election.cc:302] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce1d7fc65f9146b5907a4cdee61776c4; no voters:
I20260806 02:14:53.593051 31539 leader_election.cc:288] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:14:53.593161 31545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:14:53.593410 31545 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 1 LEADER]: Becoming Leader. State: Replica: ce1d7fc65f9146b5907a4cdee61776c4, State: Running, Role: LEADER
I20260806 02:14:53.593773 31545 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33443 } }
I20260806 02:14:53.594303 31539 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:14:53.595491 31546 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33443 } } }
I20260806 02:14:53.595635 31546 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [sys.catalog]: This master's current role is: LEADER
I20260806 02:14:53.596032 31555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:14:53.595993 31547 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce1d7fc65f9146b5907a4cdee61776c4. Latest consensus state: current_term: 1 leader_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33443 } } }
I20260806 02:14:53.596226 31547 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [sys.catalog]: This master's current role is: LEADER
I20260806 02:14:53.599005 31555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:14:53.599491 31539 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 02:14:53.608630 31555 catalog_manager.cc:1347] Generated new cluster ID: 4ed36c96b2a54f6b855eec96c9b2c2c1
I20260806 02:14:53.608700 31555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:14:53.627442 31555 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:14:53.628397 31555 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:14:53.633366 31555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4: Generated new TSK 0
I20260806 02:14:53.634052 31555 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:14:54.403205 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.70.84:51435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33443
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:14:54.572961 31586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:14:54.573400 31586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:14:54.573556 31586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:14:54.580430 31586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:14:54.580566 31586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:14:54.580687 31586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:14:54.589015 31586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.70.84:51435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:14:54.590384 31586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:14:54.591782 31586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:14:54.603924 31593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:54.603956 31594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:54.604506 31596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:54.605237 31586 server_base.cc:1034] running on GCE node
I20260806 02:14:54.605690 31586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:14:54.606421 31586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:14:54.607623 31586 hybrid_clock.cc:648] HybridClock initialized: now 1785982494607598 us; error 40 us; skew 500 ppm
Aug 06 02:14:54 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982494, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:54.611011 31586 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:14:54.612031 31586 webserver.cc:460] Webserver started at http://127.18.70.65:35587/ using document root <none> and password file <none>
I20260806 02:14:54.612733 31586 fs_manager.cc:359] Metadata directory not provided
I20260806 02:14:54.612824 31586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:14:54.613111 31586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 02:14:54 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982494, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:54.629585 31586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "b2b39f23963745159da2d9ebd7e42f9f"
format_stamp: "Formatted at 2026-08-06 02:14:54 on dist-test-slave-6k22"
server_key: "c2a4635b25de176d70038bd574b439a5"
server_key_iv: "c732d32688a9dcdccf2ceb588bb25578"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:54.630646 31586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "b2b39f23963745159da2d9ebd7e42f9f"
format_stamp: "Formatted at 2026-08-06 02:14:54 on dist-test-slave-6k22"
server_key: "c2a4635b25de176d70038bd574b439a5"
server_key_iv: "c732d32688a9dcdccf2ceb588bb25578"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:54.654579 31586 fs_manager.cc:691] Time spent creating directory manager: real 0.024s user 0.004s sys 0.000s
I20260806 02:14:54.679504 31605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:54.681231 31586 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 02:14:54.681443 31586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "b2b39f23963745159da2d9ebd7e42f9f"
format_stamp: "Formatted at 2026-08-06 02:14:54 on dist-test-slave-6k22"
server_key: "c2a4635b25de176d70038bd574b439a5"
server_key_iv: "c732d32688a9dcdccf2ceb588bb25578"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:54.681696 31586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:14:54.715322 31586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:14:54.719079 31586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:14:54.719339 31586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:14:54.720142 31586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:14:54.721309 31586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:14:54.721432 31586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:54.721577 31586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:14:54.721665 31586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:54.735864 31586 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:42999
I20260806 02:14:54.737200 31586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:14:54.744220 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 31586
I20260806 02:14:54.744361 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:14:54.744714 18713 external_mini_cluster.cc:1428] Setting key e88e49710ff43d475a29a1ff5e9e138f
I20260806 02:14:54.738965 31723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:42999 every 8 connection(s)
Aug 06 02:14:54 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982494, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:14:54.758581 31727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:54.741505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:33443 (local address 127.18.70.65:43991)
0806 02:14:54.742773 (+ 1268us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:54.742832 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:54.743932 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:54.745844 (+ 1912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:54.745854 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:54.746211 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:14:54.746922 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:54.746943 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:54.748156 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:54.748161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:54.749565 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:54.749576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:54.750357 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:54.750361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:54.750549 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:54.751261 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:14:54.751281 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:14:54.753108 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:14:54.755338 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:54.755344 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:54.755358 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:54.755753 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:54.756089 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:54.756092 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:54.756094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:54.756256 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:54.756722 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:14:54.756729 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:14:54.757029 (+ 300us) client_negotiation.cc:769] Sending connection context
0806 02:14:54.757224 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 02:14:54.757554 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":997,"thread_start_us":155,"threads_started":1}
I20260806 02:14:54.759322 31726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:54.741014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:43991 (local address 127.18.70.126:33443)
0806 02:14:54.744550 (+ 3536us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:54.744555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:54.744590 (+ 35us) server_negotiation.cc:407] Connection header received
0806 02:14:54.744640 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:54.744643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:54.744712 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:54.744925 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:54.747107 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:54.747986 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:54.749714 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:54.750065 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:54.750187 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:54.753295 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:54.753299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:54.753302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:54.753326 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:54.755193 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:54.755881 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:54.755884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:54.755887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:54.755976 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:54.756363 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:54.756366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:54.756368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:54.756570 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:54.756667 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:54.759027 (+ 2360us) server_negotiation.cc:299] Negotiation successful
0806 02:14:54.759152 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3429,"thread_start_us":82,"threads_started":1}
I20260806 02:14:54.761045 31724 heartbeater.cc:347] Connected to a master server at 127.18.70.126:33443
I20260806 02:14:54.761272 31724 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:54.761806 31724 heartbeater.cc:511] Master 127.18.70.126:33443 requested a full tablet report, sending...
I20260806 02:14:54.763609 31502 ts_manager.cc:194] Registered new tserver with Master: b2b39f23963745159da2d9ebd7e42f9f (127.18.70.65:42999)
I20260806 02:14:54.765288 31502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:43991
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
Aug 06 02:14:54 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785982494, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:14:54.825592 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.70.84:51435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33443
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:14:54.981730 31730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:14:54.982120 31730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:14:54.982236 31730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:14:54.988427 31730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:14:54.988565 31730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:14:54.988715 31730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:14:54.996588 31730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.70.84:51435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:14:54.998162 31730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:14:54.999610 31730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:14:55.012676 31739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:55.012686 31737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:55.013964 31730 server_base.cc:1034] running on GCE node
W20260806 02:14:55.019285 31736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:55.019901 31730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:14:55.020694 31730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:14:55.021855 31730 hybrid_clock.cc:648] HybridClock initialized: now 1785982495021791 us; error 81 us; skew 500 ppm
Aug 06 02:14:55 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982495, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:55.028373 31730 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:14:55.029486 31730 webserver.cc:460] Webserver started at http://127.18.70.66:43565/ using document root <none> and password file <none>
I20260806 02:14:55.030251 31730 fs_manager.cc:359] Metadata directory not provided
I20260806 02:14:55.030328 31730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:14:55.030565 31730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 02:14:55 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982495, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:55.050787 31730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "706825a2ac8d46b99156e51aa592a996"
format_stamp: "Formatted at 2026-08-06 02:14:55 on dist-test-slave-6k22"
server_key: "e1a9882d21f6f90a7fe2852f5b922334"
server_key_iv: "f07fc5fe56d9a7dd8c75f9611e3be8ca"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:55.051671 31730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "706825a2ac8d46b99156e51aa592a996"
format_stamp: "Formatted at 2026-08-06 02:14:55 on dist-test-slave-6k22"
server_key: "e1a9882d21f6f90a7fe2852f5b922334"
server_key_iv: "f07fc5fe56d9a7dd8c75f9611e3be8ca"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:55.057245 31730 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 02:14:55.081549 31747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:55.083678 31730 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.004s sys 0.000s
I20260806 02:14:55.083950 31730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "706825a2ac8d46b99156e51aa592a996"
format_stamp: "Formatted at 2026-08-06 02:14:55 on dist-test-slave-6k22"
server_key: "e1a9882d21f6f90a7fe2852f5b922334"
server_key_iv: "f07fc5fe56d9a7dd8c75f9611e3be8ca"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:55.084260 31730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:14:55.110395 31730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:14:55.114770 31730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:14:55.115056 31730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:14:55.123005 31730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:14:55.124192 31730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:14:55.124301 31730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:55.124387 31730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:14:55.124435 31730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:55.134684 31730 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:36851
I20260806 02:14:55.136009 31730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:14:55.140661 31863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:36851 every 8 connection(s)
I20260806 02:14:55.144812 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 31730
I20260806 02:14:55.144944 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:14:55.145236 18713 external_mini_cluster.cc:1428] Setting key cb83a2070bdcd32055c8af0571b8091e
Aug 06 02:14:55 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982495, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:14:55.159878 31726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:55.145725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:53587 (local address 127.18.70.126:33443)
0806 02:14:55.145858 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:55.145863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:55.146702 (+ 839us) server_negotiation.cc:407] Connection header received
0806 02:14:55.148090 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:55.148096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:55.148159 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:55.148214 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:55.149892 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:55.150620 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:55.152297 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:55.152646 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:55.152719 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:55.155734 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:55.155738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:55.155740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:55.155765 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:55.157504 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:55.158107 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:55.158110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:55.158112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:55.158193 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:55.158598 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:55.158601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:55.158603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:55.158814 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:55.158892 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:55.159514 (+ 622us) server_negotiation.cc:299] Negotiation successful
0806 02:14:55.159680 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 02:14:55.161345 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:55.143480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:33443 (local address 127.18.70.66:53587)
0806 02:14:55.146468 (+ 2988us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:55.146528 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:55.147838 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:55.148677 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:55.148689 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:55.149053 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:14:55.149732 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:55.149750 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:55.150796 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:55.150799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:55.152157 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:55.152168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:55.152913 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:55.152916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:55.153106 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:55.153800 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:14:55.153820 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:14:55.155508 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:14:55.157653 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:55.157658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:55.157672 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:55.157982 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:55.158295 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:55.158298 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:55.158300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:55.158490 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:55.158960 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:14:55.158967 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:14:55.159249 (+ 282us) client_negotiation.cc:769] Sending connection context
0806 02:14:55.160096 (+ 847us) client_negotiation.cc:240] Negotiation successful
0806 02:14:55.160398 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2759,"thread_start_us":140,"threads_started":1}
I20260806 02:14:55.162654 31864 heartbeater.cc:347] Connected to a master server at 127.18.70.126:33443
I20260806 02:14:55.162896 31864 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:55.163426 31864 heartbeater.cc:511] Master 127.18.70.126:33443 requested a full tablet report, sending...
I20260806 02:14:55.164513 31502 ts_manager.cc:194] Registered new tserver with Master: 706825a2ac8d46b99156e51aa592a996 (127.18.70.66:36851)
I20260806 02:14:55.165316 31502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:53587
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:14:55.195262 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.70.84:51435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33443
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 02:14:55 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982494, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
W20260806 02:14:55.384887 31869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:14:55.385263 31869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:14:55.385448 31869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:14:55.393870 31869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:14:55.393990 31869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:14:55.394117 31869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:14:55.401356 31869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.70.84:51435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:33443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:14:55.402439 31869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:14:55.403483 31869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:14:55.415841 31874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:55.416159 31869 server_base.cc:1034] running on GCE node
W20260806 02:14:55.416554 31875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:55.416904 31877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:55.417447 31869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:14:55.418169 31869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:14:55.421442 31869 hybrid_clock.cc:648] HybridClock initialized: now 1785982495421372 us; error 77 us; skew 500 ppm
Aug 06 02:14:55 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982495, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:55.425032 31869 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:14:55.426113 31869 webserver.cc:460] Webserver started at http://127.18.70.67:43689/ using document root <none> and password file <none>
I20260806 02:14:55.426798 31869 fs_manager.cc:359] Metadata directory not provided
I20260806 02:14:55.426873 31869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:14:55.427111 31869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 02:14:55 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982495, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:55.463145 31869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "d9653fecb9214d62aa4f808ae9104301"
format_stamp: "Formatted at 2026-08-06 02:14:55 on dist-test-slave-6k22"
server_key: "edcba7c87411b49add34c25278f2e381"
server_key_iv: "21a2ea3262c94c51e3529632a3d0ba38"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:55.468956 31869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "d9653fecb9214d62aa4f808ae9104301"
format_stamp: "Formatted at 2026-08-06 02:14:55 on dist-test-slave-6k22"
server_key: "edcba7c87411b49add34c25278f2e381"
server_key_iv: "21a2ea3262c94c51e3529632a3d0ba38"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:55.489001 31869 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.004s sys 0.000s
I20260806 02:14:55.507274 31884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:55.509290 31869 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 02:14:55.509428 31869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "d9653fecb9214d62aa4f808ae9104301"
format_stamp: "Formatted at 2026-08-06 02:14:55 on dist-test-slave-6k22"
server_key: "edcba7c87411b49add34c25278f2e381"
server_key_iv: "21a2ea3262c94c51e3529632a3d0ba38"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:55.509572 31869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:14:55.544924 31869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:14:55.548787 31869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:14:55.549057 31869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:14:55.549799 31869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:14:55.550868 31869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:14:55.550971 31869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:55.551069 31869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:14:55.551132 31869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:55.561546 31869 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:35883
I20260806 02:14:55.562706 31869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:14:55.566020 32000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:35883 every 8 connection(s)
I20260806 02:14:55.567451 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 31869
I20260806 02:14:55.567588 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:14:55.567888 18713 external_mini_cluster.cc:1428] Setting key c7e18de25e3b9eb0f71ee87852d8c9ab
Aug 06 02:14:55 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982495, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:14:55.597669 31726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:55.570156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:35543 (local address 127.18.70.126:33443)
0806 02:14:55.570280 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:55.570285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:55.580247 (+ 9962us) server_negotiation.cc:407] Connection header received
0806 02:14:55.581609 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:55.581612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:55.581672 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:55.581772 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:55.585231 (+ 3459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:55.586060 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:55.587997 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:55.588381 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:55.588465 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:55.591869 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:55.591873 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:55.591876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:55.591902 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:55.593662 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:55.594323 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:55.594327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:55.594329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:55.594409 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:55.596370 (+ 1961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:55.596374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:55.596375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:55.596598 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:55.596698 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:55.597388 (+ 690us) server_negotiation.cc:299] Negotiation successful
0806 02:14:55.597509 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:14:55.599594 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:55.569030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:33443 (local address 127.18.70.67:35543)
0806 02:14:55.580013 (+ 10983us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:55.580099 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:55.581341 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:55.582145 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:55.582156 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:55.582584 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:14:55.585042 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:55.585064 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:55.586234 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:55.586237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:55.587839 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:55.587851 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:55.588739 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:55.588743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:55.588942 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:55.589649 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:14:55.589671 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:14:55.591685 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:14:55.593804 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:55.593810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:55.593827 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:55.594212 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:55.594509 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:55.594512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:55.594514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:55.596254 (+ 1740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:55.596768 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:14:55.596774 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:14:55.597075 (+ 301us) client_negotiation.cc:769] Sending connection context
0806 02:14:55.598112 (+ 1037us) client_negotiation.cc:240] Negotiation successful
0806 02:14:55.598500 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10660,"thread_start_us":165,"threads_started":1}
I20260806 02:14:55.601068 32001 heartbeater.cc:347] Connected to a master server at 127.18.70.126:33443
I20260806 02:14:55.601346 32001 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:55.601903 32001 heartbeater.cc:511] Master 127.18.70.126:33443 requested a full tablet report, sending...
I20260806 02:14:55.602973 31502 ts_manager.cc:194] Registered new tserver with Master: d9653fecb9214d62aa4f808ae9104301 (127.18.70.67:35883)
I20260806 02:14:55.603736 31502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:35543
I20260806 02:14:55.607844 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:14:55.608811 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 31586
I20260806 02:14:55.627170 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 31730
I20260806 02:14:55.635771 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 31869
I20260806 02:14:55.648623 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 31467
I20260806 02:14:55.656570 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:33443
--webserver_interface=127.18.70.126
--webserver_port=41333
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:33443
--ranger_config_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.70.84:51435/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.70.84:51435/kms with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:14:55.767436 32005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:14:55.767776 32005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:14:55.767843 32005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:14:55.771618 32005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:14:55.771667 32005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:14:55.771683 32005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:14:55.771698 32005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:14:55.771713 32005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:14:55.777009 32005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.70.84:51435/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:33443
--ranger_config_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:33443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=41333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:14:55.777972 32005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:14:55.778951 32005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:14:55.787245 32011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:55.787236 32010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:55.787246 32013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:55.787436 32005 server_base.cc:1034] running on GCE node
I20260806 02:14:55.788043 32005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:14:55.789108 32005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:14:55.790297 32005 hybrid_clock.cc:648] HybridClock initialized: now 1785982495790282 us; error 40 us; skew 500 ppm
Aug 06 02:14:55 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982495, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:55.792969 32005 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:14:55.793843 32005 webserver.cc:460] Webserver started at http://127.18.70.126:41333/ using document root <none> and password file <none>
I20260806 02:14:55.794368 32005 fs_manager.cc:359] Metadata directory not provided
I20260806 02:14:55.794456 32005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 02:14:55 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982495, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:55.850526 32005 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 02:14:55.852843 32022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:55.854121 32005 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:14:55.854270 32005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "ce1d7fc65f9146b5907a4cdee61776c4"
format_stamp: "Formatted at 2026-08-06 02:14:53 on dist-test-slave-6k22"
server_key: "81f22867b3b56063c4227a70cf27eac2"
server_key_iv: "a1a5c826e15f80b39494c31dc1095f6b"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:55.854844 32005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:14:55.870075 32005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:14:55.872546 32005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:14:55.872741 32005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:14:55.879913 32005 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:33443
I20260806 02:14:55.879948 32075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:33443 every 8 connection(s)
I20260806 02:14:55.880947 32005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:14:55.883652 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 32005
I20260806 02:14:55.884151 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.70.84:51435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:42999
--local_ip_for_outbound_sockets=127.18.70.65
--tserver_master_addrs=127.18.70.126:33443
--webserver_port=35587
--webserver_interface=127.18.70.65
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.70.84:51435/kms with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:14:55.885542 32078 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:14:55.887245 32078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4: Bootstrap starting.
I20260806 02:14:55.903306 32078 log.cc:824] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4: Log is configured to *not* fsync() on all Append() calls
I20260806 02:14:55.909453 32078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:14:55.909838 32078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4: Bootstrap complete.
I20260806 02:14:55.913923 32078 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33443 } }
I20260806 02:14:55.914297 32078 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1d7fc65f9146b5907a4cdee61776c4, State: Initialized, Role: FOLLOWER
I20260806 02:14:55.914815 32078 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33443 } }
I20260806 02:14:55.914950 32078 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:14:55.915030 32078 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:14:55.915140 32078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 1 FOLLOWER]: Advancing to term 2
I20260806 02:14:55.916925 32078 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33443 } }
I20260806 02:14:55.917239 32078 leader_election.cc:302] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce1d7fc65f9146b5907a4cdee61776c4; no voters:
I20260806 02:14:55.917531 32078 leader_election.cc:288] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 02:14:55.917672 32082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 02:14:55.917886 32082 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [term 2 LEADER]: Becoming Leader. State: Replica: ce1d7fc65f9146b5907a4cdee61776c4, State: Running, Role: LEADER
I20260806 02:14:55.918249 32082 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33443 } }
I20260806 02:14:55.918720 32078 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:14:55.919709 32083 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33443 } } }
I20260806 02:14:55.919832 32083 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [sys.catalog]: This master's current role is: LEADER
I20260806 02:14:55.920593 32084 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce1d7fc65f9146b5907a4cdee61776c4. Latest consensus state: current_term: 2 leader_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce1d7fc65f9146b5907a4cdee61776c4" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33443 } } }
I20260806 02:14:55.920720 32084 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4 [sys.catalog]: This master's current role is: LEADER
I20260806 02:14:55.920967 32091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:14:55.920996 32078 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 02:14:55.927488 32091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:14:55.928318 32091 catalog_manager.cc:1259] Loaded cluster ID: 4ed36c96b2a54f6b855eec96c9b2c2c1
I20260806 02:14:55.928376 32091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:14:55.931063 32091 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:14:55.931967 32091 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4: Loaded TSK: 0
I20260806 02:14:55.933317 32091 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 02:14:56.036258 32080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:14:56.036701 32080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:14:56.036782 32080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:14:56.043097 32080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:14:56.043222 32080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:14:56.043354 32080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:14:56.051694 32080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.70.84:51435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:42999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=35587
--tserver_master_addrs=127.18.70.126:33443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:14:56.053076 32080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:14:56.054428 32080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:14:56.067732 32119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:56.067744 32117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:56.067701 32116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:56.069495 32080 server_base.cc:1034] running on GCE node
I20260806 02:14:56.070097 32080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:14:56.070925 32080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:14:56.072116 32080 hybrid_clock.cc:648] HybridClock initialized: now 1785982496072029 us; error 99 us; skew 500 ppm
Aug 06 02:14:56 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982496, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:56.076332 32080 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:14:56.080330 32080 webserver.cc:460] Webserver started at http://127.18.70.65:35587/ using document root <none> and password file <none>
I20260806 02:14:56.081117 32080 fs_manager.cc:359] Metadata directory not provided
I20260806 02:14:56.081276 32080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 02:14:56 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982496, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:56.119848 32080 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.003s sys 0.000s
I20260806 02:14:56.123015 32127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:56.124609 32080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:14:56.124720 32080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "b2b39f23963745159da2d9ebd7e42f9f"
format_stamp: "Formatted at 2026-08-06 02:14:54 on dist-test-slave-6k22"
server_key: "c2a4635b25de176d70038bd574b439a5"
server_key_iv: "c732d32688a9dcdccf2ceb588bb25578"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:56.125413 32080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:14:56.170974 32080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:14:56.175277 32080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:14:56.175544 32080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:14:56.176352 32080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:14:56.177621 32080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:14:56.177696 32080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:56.177748 32080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:14:56.177809 32080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:56.202555 32080 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:42999
I20260806 02:14:56.202623 32242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:42999 every 8 connection(s)
I20260806 02:14:56.204181 32080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:14:56.213402 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 32080
I20260806 02:14:56.213908 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.70.84:51435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:36851
--local_ip_for_outbound_sockets=127.18.70.66
--tserver_master_addrs=127.18.70.126:33443
--webserver_port=43565
--webserver_interface=127.18.70.66
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.70.84:51435/kms with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 02:14:56 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982496, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:14:56.227450 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:56.209413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:33443 (local address 127.18.70.65:50483)
0806 02:14:56.209916 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:56.209956 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:56.210842 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:56.212026 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:56.212051 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:56.212384 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:14:56.212929 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:56.212953 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.216169 (+ 3216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.216173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:56.217461 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:56.217473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.218794 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.218799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:56.218993 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:56.219842 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:14:56.219863 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:14:56.221455 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:14:56.224218 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:56.224245 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:56.224259 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:56.224510 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:56.224914 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:56.224917 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:56.224919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:56.225051 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:56.225692 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:14:56.225698 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:14:56.225938 (+ 240us) client_negotiation.cc:769] Sending connection context
0806 02:14:56.226134 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 02:14:56.226457 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":165,"threads_started":1}
I20260806 02:14:56.227438 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:56.209149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:50483 (local address 127.18.70.126:33443)
0806 02:14:56.209635 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:56.209658 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:56.210121 (+ 463us) server_negotiation.cc:407] Connection header received
0806 02:14:56.211276 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:56.211286 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:56.211657 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:56.211958 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:56.213130 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.214747 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.217598 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.218307 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.218484 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:56.221662 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:56.221668 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:56.221683 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:56.221712 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:56.224032 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:56.224695 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:56.224700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:56.224704 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:56.224812 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:56.225143 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:56.225149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:56.225154 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:56.225564 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:56.225720 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:56.226242 (+ 522us) server_negotiation.cc:299] Negotiation successful
0806 02:14:56.226479 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":137,"threads_started":1}
I20260806 02:14:56.230168 32243 heartbeater.cc:347] Connected to a master server at 127.18.70.126:33443
I20260806 02:14:56.230410 32243 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:56.230940 32243 heartbeater.cc:511] Master 127.18.70.126:33443 requested a full tablet report, sending...
W20260806 02:14:56.233304 32243 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:33443 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 02:14:56.233429 32243 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:56.233589 32243 heartbeater.cc:511] Master 127.18.70.126:33443 requested a full tablet report, sending...
I20260806 02:14:56.234560 32243 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:56.234750 32243 heartbeater.cc:511] Master 127.18.70.126:33443 requested a full tablet report, sending...
W20260806 02:14:56.235450 32243 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 02:14:56.393193 32247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:14:56.393679 32247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:14:56.393806 32247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:14:56.400110 32247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:14:56.400202 32247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:14:56.400317 32247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:14:56.408286 32247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.70.84:51435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:36851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=43565
--tserver_master_addrs=127.18.70.126:33443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:14:56.409883 32247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:14:56.411356 32247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:14:56.424566 32256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:56.424755 32253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:56.425190 32254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:56.425688 32247 server_base.cc:1034] running on GCE node
I20260806 02:14:56.426119 32247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:14:56.426761 32247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:14:56.427939 32247 hybrid_clock.cc:648] HybridClock initialized: now 1785982496427913 us; error 44 us; skew 500 ppm
Aug 06 02:14:56 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982496, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:56.431322 32247 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:14:56.432678 32247 webserver.cc:460] Webserver started at http://127.18.70.66:43565/ using document root <none> and password file <none>
I20260806 02:14:56.433416 32247 fs_manager.cc:359] Metadata directory not provided
I20260806 02:14:56.433517 32247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 02:14:56 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982496, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:56.461511 32247 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.000s
I20260806 02:14:56.466256 32265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:56.467921 32247 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 02:14:56.468050 32247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "706825a2ac8d46b99156e51aa592a996"
format_stamp: "Formatted at 2026-08-06 02:14:55 on dist-test-slave-6k22"
server_key: "e1a9882d21f6f90a7fe2852f5b922334"
server_key_iv: "f07fc5fe56d9a7dd8c75f9611e3be8ca"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:56.468640 32247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:14:56.483541 32247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:14:56.486862 32247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:14:56.487113 32247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:14:56.487905 32247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:14:56.488992 32247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:14:56.489079 32247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:56.489151 32247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:14:56.489189 32247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:56.500257 32247 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:36851
I20260806 02:14:56.500586 32385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:36851 every 8 connection(s)
I20260806 02:14:56.501356 32247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:14:56.506278 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 32247
I20260806 02:14:56.506769 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.70.84:51435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:35883
--local_ip_for_outbound_sockets=127.18.70.67
--tserver_master_addrs=127.18.70.126:33443
--webserver_port=43689
--webserver_interface=127.18.70.67
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.70.84:51435/kms with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 02:14:56 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982496, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:14:56.523619 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:56.503311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:36989 (local address 127.18.70.126:33443)
0806 02:14:56.503448 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:56.503452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:56.504413 (+ 961us) server_negotiation.cc:407] Connection header received
0806 02:14:56.505628 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:56.505631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:56.505689 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:56.505766 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:56.507254 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.508097 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.509917 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.510305 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.510424 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:56.517612 (+ 7188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:56.517618 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:56.517622 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:56.517674 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:56.519830 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:56.520624 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:56.520630 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:56.520633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:56.520789 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:56.521239 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:56.521245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:56.521249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:56.521520 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:56.521625 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:56.523279 (+ 1654us) server_negotiation.cc:299] Negotiation successful
0806 02:14:56.523414 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 02:14:56.524048 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:56.503790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:33443 (local address 127.18.70.66:36989)
0806 02:14:56.504264 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:56.504305 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:56.505382 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:56.505951 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:56.505992 (+ 41us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:56.506343 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:14:56.507049 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:56.507072 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.508258 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.508264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:56.509692 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:56.509706 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.510485 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.510491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:56.510703 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:56.515701 (+ 4998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:14:56.515723 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:14:56.517420 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:14:56.520044 (+ 2624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:56.520052 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:56.520070 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:56.520484 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:56.520909 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:56.520913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:56.520918 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:56.521115 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:56.521686 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:14:56.521693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:14:56.522045 (+ 352us) client_negotiation.cc:769] Sending connection context
0806 02:14:56.522293 (+ 248us) client_negotiation.cc:240] Negotiation successful
0806 02:14:56.522706 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":149,"threads_started":1}
I20260806 02:14:56.525381 32386 heartbeater.cc:347] Connected to a master server at 127.18.70.126:33443
I20260806 02:14:56.525612 32386 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:56.526127 32386 heartbeater.cc:511] Master 127.18.70.126:33443 requested a full tablet report, sending...
W20260806 02:14:56.527881 32386 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:33443 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 02:14:56.527994 32386 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:56.528133 32386 heartbeater.cc:511] Master 127.18.70.126:33443 requested a full tablet report, sending...
I20260806 02:14:56.528893 32386 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:56.529045 32386 heartbeater.cc:511] Master 127.18.70.126:33443 requested a full tablet report, sending...
W20260806 02:14:56.529721 32386 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 02:14:56.651419 32389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:14:56.651799 32389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:14:56.651937 32389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:14:56.657796 32389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:14:56.657872 32389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:14:56.657963 32389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:14:56.664427 32389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.70.84:51435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:35883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=43689
--tserver_master_addrs=127.18.70.126:33443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:14:56.665808 32389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:14:56.666975 32389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:14:56.677913 32395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:56.678244 32394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:14:56.678889 32397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:14:56.678894 32389 server_base.cc:1034] running on GCE node
I20260806 02:14:56.679454 32389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:14:56.680156 32389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:14:56.681375 32389 hybrid_clock.cc:648] HybridClock initialized: now 1785982496681338 us; error 43 us; skew 500 ppm
Aug 06 02:14:56 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982496, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:56.685174 32389 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:14:56.686465 32389 webserver.cc:460] Webserver started at http://127.18.70.67:43689/ using document root <none> and password file <none>
I20260806 02:14:56.687217 32389 fs_manager.cc:359] Metadata directory not provided
I20260806 02:14:56.687383 32389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 02:14:56 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982496, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:56.706697 32389 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:14:56.709523 32405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:56.710990 32389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 02:14:56.711125 32389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "d9653fecb9214d62aa4f808ae9104301"
format_stamp: "Formatted at 2026-08-06 02:14:55 on dist-test-slave-6k22"
server_key: "edcba7c87411b49add34c25278f2e381"
server_key_iv: "21a2ea3262c94c51e3529632a3d0ba38"
server_key_version: "kuduclusterkey@0"
I20260806 02:14:56.711779 32389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:14:56.733806 32389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:14:56.737010 32389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:14:56.737242 32389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:14:56.737905 32389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:14:56.738875 32389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:14:56.738924 32389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:56.738965 32389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:14:56.739012 32389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:14:56.750144 32389 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:35883
I20260806 02:14:56.750231 32531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:35883 every 8 connection(s)
I20260806 02:14:56.751223 32389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:14:56.756846 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 32389
Aug 06 02:14:56 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982496, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:14:56.768270 32537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:56.757528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40106 (local address 127.18.70.126:33443)
0806 02:14:56.757828 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:56.757835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:56.758133 (+ 298us) server_negotiation.cc:407] Connection header received
0806 02:14:56.758247 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:56.758250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:56.758297 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:56.758420 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:56.759246 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.759879 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.761121 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.761494 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.761621 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:56.763136 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:56.763142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:56.763147 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:56.763197 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:56.765115 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:56.765716 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:56.765720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:56.765721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:56.765825 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:56.766130 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:56.766133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:56.766134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:56.766334 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:56.766535 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:56.767967 (+ 1432us) server_negotiation.cc:299] Negotiation successful
0806 02:14:56.768091 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":125,"threads_started":1}
I20260806 02:14:56.769232 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:56.753202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:39345 (local address 127.18.70.126:33443)
0806 02:14:56.753365 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:56.753370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:56.754225 (+ 855us) server_negotiation.cc:407] Connection header received
0806 02:14:56.755165 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:56.755169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:56.755224 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:56.755339 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:56.757300 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.758157 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.759905 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.760151 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.760248 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:56.762909 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:56.762913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:56.762915 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:56.762944 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:56.764725 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:56.765966 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:56.765968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:56.765970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:56.766050 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:56.766990 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:56.766993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:56.766995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:56.767195 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:56.767263 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:56.768982 (+ 1719us) server_negotiation.cc:299] Negotiation successful
0806 02:14:56.769092 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 02:14:56.771431 32534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:56.753571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:33443 (local address 127.18.70.67:39345)
0806 02:14:56.754051 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:56.754103 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:56.754948 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:56.755508 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:56.755515 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:56.755963 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:14:56.756454 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:56.756469 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.758321 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.758325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:56.759779 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:56.759792 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:56.760305 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:56.760309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:56.760469 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:56.761027 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:14:56.761043 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:14:56.762693 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:14:56.765253 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:56.765259 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:56.765274 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:56.765607 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:56.766618 (+ 1011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:14:56.766674 (+ 56us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:14:56.766676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:14:56.766867 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:14:56.767351 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:14:56.767357 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:14:56.767688 (+ 331us) client_negotiation.cc:769] Sending connection context
0806 02:14:56.770029 (+ 2341us) client_negotiation.cc:240] Negotiation successful
0806 02:14:56.770375 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":142,"threads_started":1}
I20260806 02:14:56.772899 32532 heartbeater.cc:347] Connected to a master server at 127.18.70.126:33443
I20260806 02:14:56.773137 32532 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:56.773684 32532 heartbeater.cc:511] Master 127.18.70.126:33443 requested a full tablet report, sending...
I20260806 02:14:56.775162 32039 ts_manager.cc:194] Registered new tserver with Master: d9653fecb9214d62aa4f808ae9104301 (127.18.70.67:35883)
I20260806 02:14:56.776722 32039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:39345
Aug 06 02:14:57 dist-test-slave-6k22 krb5kdc[31005](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785982497, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:14:57.235649 32243 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:57.235951 32243 heartbeater.cc:511] Master 127.18.70.126:33443 requested a full tablet report, sending...
I20260806 02:14:57.236680 32039 ts_manager.cc:194] Registered new tserver with Master: b2b39f23963745159da2d9ebd7e42f9f (127.18.70.65:42999)
I20260806 02:14:57.237347 32039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:50483
Aug 06 02:14:57 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982497, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
I20260806 02:14:57.529924 32386 heartbeater.cc:464] Registering TS with master...
I20260806 02:14:57.530236 32386 heartbeater.cc:511] Master 127.18.70.126:33443 requested a full tablet report, sending...
I20260806 02:14:57.530979 32039 ts_manager.cc:194] Registered new tserver with Master: 706825a2ac8d46b99156e51aa592a996 (127.18.70.66:36851)
I20260806 02:14:57.531785 32039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:36989
I20260806 02:14:57.541119 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:14:57.550904 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:57.542735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40112 (local address 127.18.70.126:33443)
0806 02:14:57.543010 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:57.543013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:57.543041 (+ 28us) server_negotiation.cc:407] Connection header received
0806 02:14:57.543164 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:57.543166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:57.543204 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:57.543277 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:14:57.544157 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.544694 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.545805 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.546027 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.546101 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:57.547704 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:14:57.547707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:14:57.547709 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:14:57.547731 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:14:57.549204 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:57.549705 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:57.549708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:57.549709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:57.549784 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:14:57.550083 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:14:57.550085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:14:57.550086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:14:57.550261 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:14:57.550373 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:57.550646 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 02:14:57.550745 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":90,"threads_started":1}
I20260806 02:14:57.554751 32039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:14:57.559643 32039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:14:57.581080 32553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:57.571827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:36851 (local address 127.0.0.1:36422)
0806 02:14:57.573695 (+ 1868us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:57.573709 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:57.573814 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:57.576227 (+ 2413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:57.576230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:57.576249 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:14:57.576385 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:57.576391 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.577786 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.577789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:57.579165 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:57.579176 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.580530 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.580534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:57.580652 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:57.580695 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 02:14:57.580810 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 02:14:57.580932 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1815,"thread_start_us":93,"threads_started":1}
I20260806 02:14:57.583730 32555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:57.572421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:42999 (local address 127.0.0.1:43614)
0806 02:14:57.576540 (+ 4119us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:57.576554 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:57.576728 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:57.578201 (+ 1473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:57.578204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:57.578221 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:14:57.578311 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:57.578317 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.579874 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.579877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:57.581932 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:57.581947 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.583266 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.583270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:57.583377 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:57.583401 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 02:14:57.583488 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 02:14:57.583589 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4049,"thread_start_us":84,"threads_started":1}
I20260806 02:14:57.584316 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:57.571820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36422 (local address 127.18.70.66:36851)
0806 02:14:57.575793 (+ 3973us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:57.575800 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:57.575823 (+ 23us) server_negotiation.cc:407] Connection header received
0806 02:14:57.575898 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:57.575903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:57.576094 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:57.576243 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:14:57.576533 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.577648 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.579339 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.580188 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.580307 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:57.580462 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:57.584035 (+ 3573us) server_negotiation.cc:299] Negotiation successful
0806 02:14:57.584150 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3865,"thread_start_us":72,"threads_started":1}
I20260806 02:14:57.585016 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:57.573244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43614 (local address 127.18.70.65:42999)
0806 02:14:57.573605 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:57.573613 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:57.576653 (+ 3040us) server_negotiation.cc:407] Connection header received
0806 02:14:57.577686 (+ 1033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:57.577693 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:57.577889 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:57.578020 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:14:57.578472 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.579667 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.582088 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.582903 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.583019 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:57.583157 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:57.584739 (+ 1582us) server_negotiation.cc:299] Negotiation successful
0806 02:14:57.584862 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":91,"threads_started":1}
I20260806 02:14:57.586872 32556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:57.571656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:35883 (local address 127.0.0.1:49936)
0806 02:14:57.572794 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:57.572845 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:57.572981 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:57.574571 (+ 1590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:57.574578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:57.574601 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:14:57.574741 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:57.574748 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.581815 (+ 7067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.581818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:57.582978 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:57.582989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.586533 (+ 3544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.586536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:57.586620 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:57.586642 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 02:14:57.586689 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 02:14:57.586751 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"thread_start_us":80,"threads_started":1}
I20260806 02:14:57.587688 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:57.572952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49936 (local address 127.18.70.67:35883)
0806 02:14:57.574053 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:57.574060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:57.574082 (+ 22us) server_negotiation.cc:407] Connection header received
0806 02:14:57.574166 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:57.574171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:57.574378 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:57.574905 (+ 527us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:14:57.575047 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.581695 (+ 6648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.583139 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.586391 (+ 3252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.587263 (+ 872us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:57.587408 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:57.587458 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 02:14:57.587541 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1005,"thread_start_us":68,"threads_started":1}
I20260806 02:14:57.587764 32319 tablet_service.cc:1467] Processing CreateTablet for tablet ea85a76efbd54ff987f6ca8ed01daa15 (DEFAULT_TABLE table=test-table [id=50db2b25e0394d5e96068333a3aba006]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:14:57.588058 32171 tablet_service.cc:1467] Processing CreateTablet for tablet ea85a76efbd54ff987f6ca8ed01daa15 (DEFAULT_TABLE table=test-table [id=50db2b25e0394d5e96068333a3aba006]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:14:57.589110 32171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea85a76efbd54ff987f6ca8ed01daa15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:14:57.589936 32319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea85a76efbd54ff987f6ca8ed01daa15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:14:57.590518 32461 tablet_service.cc:1467] Processing CreateTablet for tablet ea85a76efbd54ff987f6ca8ed01daa15 (DEFAULT_TABLE table=test-table [id=50db2b25e0394d5e96068333a3aba006]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:14:57.591471 32461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea85a76efbd54ff987f6ca8ed01daa15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:14:57.599700 32560 tablet_bootstrap.cc:492] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301: Bootstrap starting.
I20260806 02:14:57.602052 32560 tablet_bootstrap.cc:654] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301: Neither blocks nor log segments found. Creating new log.
I20260806 02:14:57.602432 32559 tablet_bootstrap.cc:492] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f: Bootstrap starting.
I20260806 02:14:57.604622 32560 log.cc:824] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301: Log is configured to *not* fsync() on all Append() calls
I20260806 02:14:57.605685 32559 tablet_bootstrap.cc:654] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f: Neither blocks nor log segments found. Creating new log.
I20260806 02:14:57.606820 32559 log.cc:824] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f: Log is configured to *not* fsync() on all Append() calls
I20260806 02:14:57.607900 32561 tablet_bootstrap.cc:492] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996: Bootstrap starting.
I20260806 02:14:57.608343 32560 tablet_bootstrap.cc:492] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301: No bootstrap required, opened a new log
I20260806 02:14:57.608894 32560 ts_tablet_manager.cc:1397] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260806 02:14:57.611016 32561 tablet_bootstrap.cc:654] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996: Neither blocks nor log segments found. Creating new log.
I20260806 02:14:57.611927 32561 log.cc:824] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996: Log is configured to *not* fsync() on all Append() calls
I20260806 02:14:57.613536 32560 raft_consensus.cc:348] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9653fecb9214d62aa4f808ae9104301" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35883 } } peers { permanent_uuid: "b2b39f23963745159da2d9ebd7e42f9f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 42999 } } peers { permanent_uuid: "706825a2ac8d46b99156e51aa592a996" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36851 } }
I20260806 02:14:57.613809 32560 raft_consensus.cc:374] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:14:57.613899 32560 raft_consensus.cc:729] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9653fecb9214d62aa4f808ae9104301, State: Initialized, Role: FOLLOWER
I20260806 02:14:57.614494 32560 consensus_queue.cc:260] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9653fecb9214d62aa4f808ae9104301" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35883 } } peers { permanent_uuid: "b2b39f23963745159da2d9ebd7e42f9f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 42999 } } peers { permanent_uuid: "706825a2ac8d46b99156e51aa592a996" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36851 } }
I20260806 02:14:57.616274 32561 tablet_bootstrap.cc:492] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996: No bootstrap required, opened a new log
I20260806 02:14:57.616551 32561 ts_tablet_manager.cc:1397] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 02:14:57.617723 32559 tablet_bootstrap.cc:492] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f: No bootstrap required, opened a new log
I20260806 02:14:57.617959 32559 ts_tablet_manager.cc:1397] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.000s
I20260806 02:14:57.621704 32561 raft_consensus.cc:348] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9653fecb9214d62aa4f808ae9104301" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35883 } } peers { permanent_uuid: "b2b39f23963745159da2d9ebd7e42f9f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 42999 } } peers { permanent_uuid: "706825a2ac8d46b99156e51aa592a996" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36851 } }
I20260806 02:14:57.622042 32561 raft_consensus.cc:374] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:14:57.622146 32561 raft_consensus.cc:729] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 706825a2ac8d46b99156e51aa592a996, State: Initialized, Role: FOLLOWER
I20260806 02:14:57.622298 32559 raft_consensus.cc:348] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9653fecb9214d62aa4f808ae9104301" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35883 } } peers { permanent_uuid: "b2b39f23963745159da2d9ebd7e42f9f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 42999 } } peers { permanent_uuid: "706825a2ac8d46b99156e51aa592a996" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36851 } }
I20260806 02:14:57.622601 32559 raft_consensus.cc:374] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:14:57.622701 32559 raft_consensus.cc:729] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b39f23963745159da2d9ebd7e42f9f, State: Initialized, Role: FOLLOWER
I20260806 02:14:57.622756 32561 consensus_queue.cc:260] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9653fecb9214d62aa4f808ae9104301" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35883 } } peers { permanent_uuid: "b2b39f23963745159da2d9ebd7e42f9f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 42999 } } peers { permanent_uuid: "706825a2ac8d46b99156e51aa592a996" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36851 } }
I20260806 02:14:57.623226 32560 ts_tablet_manager.cc:1428] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260806 02:14:57.623667 32559 consensus_queue.cc:260] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9653fecb9214d62aa4f808ae9104301" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35883 } } peers { permanent_uuid: "b2b39f23963745159da2d9ebd7e42f9f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 42999 } } peers { permanent_uuid: "706825a2ac8d46b99156e51aa592a996" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36851 } }
I20260806 02:14:57.623926 32561 ts_tablet_manager.cc:1428] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 02:14:57.624190 32532 heartbeater.cc:503] Master 127.18.70.126:33443 was elected leader, sending a full tablet report...
I20260806 02:14:57.624375 32386 heartbeater.cc:503] Master 127.18.70.126:33443 was elected leader, sending a full tablet report...
I20260806 02:14:57.624933 32559 ts_tablet_manager.cc:1428] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260806 02:14:57.625239 32243 heartbeater.cc:503] Master 127.18.70.126:33443 was elected leader, sending a full tablet report...
W20260806 02:14:57.709260 32244 tablet.cc:2369] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 02:14:57 dist-test-slave-6k22 krb5kdc[31005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982497, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for HTTP/127.18.70.84@KRBTEST.COM
W20260806 02:14:57.753253 32387 tablet.cc:2369] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 02:14:57.753346 32533 tablet.cc:2369] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:14:57.825279 32567 raft_consensus.cc:482] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:14:57.825531 32567 raft_consensus.cc:504] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9653fecb9214d62aa4f808ae9104301" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35883 } } peers { permanent_uuid: "b2b39f23963745159da2d9ebd7e42f9f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 42999 } } peers { permanent_uuid: "706825a2ac8d46b99156e51aa592a996" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36851 } }
I20260806 02:14:57.827056 32567 leader_election.cc:288] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9653fecb9214d62aa4f808ae9104301 (127.18.70.67:35883), 706825a2ac8d46b99156e51aa592a996 (127.18.70.66:36851)
I20260806 02:14:57.833726 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:57.828185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:50497 (local address 127.18.70.67:35883)
0806 02:14:57.828317 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:57.828322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:57.828832 (+ 510us) server_negotiation.cc:407] Connection header received
0806 02:14:57.829083 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:57.829087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:57.829144 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:57.829199 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:14:57.829689 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.830475 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.832308 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.832948 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.833048 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:57.833103 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:57.833507 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 02:14:57.833587 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 02:14:57.834415 32570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:57.827275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:35883 (local address 127.18.70.65:50497)
0806 02:14:57.828497 (+ 1222us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:57.828745 (+ 248us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:57.828926 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:57.829379 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:57.829382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:57.829400 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:14:57.829550 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:57.829557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.830628 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.830631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:57.832133 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:57.832143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.833239 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.833242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:57.833362 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:57.833386 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 02:14:57.834124 (+ 738us) client_negotiation.cc:240] Negotiation successful
0806 02:14:57.834202 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1125,"thread_start_us":124,"threads_started":1}
I20260806 02:14:57.835233 32485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea85a76efbd54ff987f6ca8ed01daa15" candidate_uuid: "b2b39f23963745159da2d9ebd7e42f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9653fecb9214d62aa4f808ae9104301" is_pre_election: true
I20260806 02:14:57.835639 32485 raft_consensus.cc:2454] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2b39f23963745159da2d9ebd7e42f9f in term 0.
I20260806 02:14:57.836277 32129 leader_election.cc:302] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2b39f23963745159da2d9ebd7e42f9f, d9653fecb9214d62aa4f808ae9104301; no voters:
I20260806 02:14:57.836587 32567 raft_consensus.cc:2781] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:14:57.836688 32567 raft_consensus.cc:482] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:14:57.836731 32567 raft_consensus.cc:3037] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:14:57.838163 32567 raft_consensus.cc:504] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9653fecb9214d62aa4f808ae9104301" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35883 } } peers { permanent_uuid: "b2b39f23963745159da2d9ebd7e42f9f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 42999 } } peers { permanent_uuid: "706825a2ac8d46b99156e51aa592a996" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36851 } }
I20260806 02:14:57.838557 32567 leader_election.cc:288] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [CANDIDATE]: Term 1 election: Requested vote from peers d9653fecb9214d62aa4f808ae9104301 (127.18.70.67:35883), 706825a2ac8d46b99156e51aa592a996 (127.18.70.66:36851)
I20260806 02:14:57.839165 32485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea85a76efbd54ff987f6ca8ed01daa15" candidate_uuid: "b2b39f23963745159da2d9ebd7e42f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9653fecb9214d62aa4f808ae9104301"
I20260806 02:14:57.839286 32485 raft_consensus.cc:3037] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:14:57.839414 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:57.827937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:35261 (local address 127.18.70.66:36851)
0806 02:14:57.828071 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:57.828076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:57.831881 (+ 3805us) server_negotiation.cc:407] Connection header received
0806 02:14:57.835195 (+ 3314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:57.835198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:57.835249 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:57.835311 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:14:57.835772 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.836504 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.838009 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.838659 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.838754 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:57.838803 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:57.839195 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 02:14:57.839291 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 02:14:57.839845 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:57.827355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:36851 (local address 127.18.70.65:35261)
0806 02:14:57.831766 (+ 4411us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:57.831803 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 02:14:57.835047 (+ 3244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:14:57.835485 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:14:57.835488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:14:57.835504 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:14:57.835637 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:57.835644 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.836675 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.836679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:57.837861 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:14:57.837873 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.838928 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.838932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:14:57.839042 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:57.839069 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 02:14:57.839638 (+ 569us) client_negotiation.cc:240] Negotiation successful
0806 02:14:57.839710 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4349,"thread_start_us":87,"threads_started":1}
I20260806 02:14:57.840672 32485 raft_consensus.cc:2454] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2b39f23963745159da2d9ebd7e42f9f in term 1.
I20260806 02:14:57.841544 32338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea85a76efbd54ff987f6ca8ed01daa15" candidate_uuid: "b2b39f23963745159da2d9ebd7e42f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "706825a2ac8d46b99156e51aa592a996"
I20260806 02:14:57.841799 32338 raft_consensus.cc:3037] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:14:57.842083 32129 leader_election.cc:302] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2b39f23963745159da2d9ebd7e42f9f, d9653fecb9214d62aa4f808ae9104301; no voters:
I20260806 02:14:57.842274 32567 raft_consensus.cc:2781] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:14:57.842499 32567 raft_consensus.cc:686] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [term 1 LEADER]: Becoming Leader. State: Replica: b2b39f23963745159da2d9ebd7e42f9f, State: Running, Role: LEADER
I20260806 02:14:57.842857 32567 consensus_queue.cc:237] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9653fecb9214d62aa4f808ae9104301" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35883 } } peers { permanent_uuid: "b2b39f23963745159da2d9ebd7e42f9f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 42999 } } peers { permanent_uuid: "706825a2ac8d46b99156e51aa592a996" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36851 } }
I20260806 02:14:57.843220 32338 raft_consensus.cc:2454] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2b39f23963745159da2d9ebd7e42f9f in term 1.
I20260806 02:14:57.841554 32339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea85a76efbd54ff987f6ca8ed01daa15" candidate_uuid: "b2b39f23963745159da2d9ebd7e42f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "706825a2ac8d46b99156e51aa592a996" is_pre_election: true
I20260806 02:14:57.843642 32339 raft_consensus.cc:2362] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b2b39f23963745159da2d9ebd7e42f9f in term 1. Re-sending same reply.
I20260806 02:14:57.846290 32037 catalog_manager.cc:5495] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f reported cstate change: term changed from 0 to 1, leader changed from <none> to b2b39f23963745159da2d9ebd7e42f9f (127.18.70.65). New cstate: current_term: 1 leader_uuid: "b2b39f23963745159da2d9ebd7e42f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9653fecb9214d62aa4f808ae9104301" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2b39f23963745159da2d9ebd7e42f9f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 42999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "706825a2ac8d46b99156e51aa592a996" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36851 } health_report { overall_health: UNKNOWN } } }
I20260806 02:14:57.872377 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:57.867084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43626 (local address 127.18.70.65:42999)
0806 02:14:57.867209 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 02:14:57.867214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:14:57.867228 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:14:57.867309 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:14:57.867312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:14:57.867373 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:14:57.867473 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:14:57.867930 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.868705 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.870004 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:14:57.870337 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:14:57.870441 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:14:57.870876 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:14:57.871423 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:14:57.871545 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 02:14:57.872125 (+ 580us) server_negotiation.cc:299] Negotiation successful
0806 02:14:57.872186 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 02:14:57.879246 32339 raft_consensus.cc:1261] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996 [term 1 FOLLOWER]: Refusing update from remote peer b2b39f23963745159da2d9ebd7e42f9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:14:57.880101 32567 consensus_queue.cc:1035] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "706825a2ac8d46b99156e51aa592a996" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 36851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:14:57.887475 32485 raft_consensus.cc:1261] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301 [term 1 FOLLOWER]: Refusing update from remote peer b2b39f23963745159da2d9ebd7e42f9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:14:57.888324 32567 consensus_queue.cc:1035] T ea85a76efbd54ff987f6ca8ed01daa15 P b2b39f23963745159da2d9ebd7e42f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9653fecb9214d62aa4f808ae9104301" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:14:57.898269 32578 mvcc.cc:205] Tried to move back new op lower bound from 7315384311304466432 to 7315384311167606784. Current Snapshot: MvccSnapshot[applied={T|T < 7315384311304466432}]
I20260806 02:14:57.904392 32576 mvcc.cc:205] Tried to move back new op lower bound from 7315384311304466432 to 7315384311167606784. Current Snapshot: MvccSnapshot[applied={T|T < 7315384311304466432}]
I20260806 02:14:57.909413 32037 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:14:57.909610 32037 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:14:57.912325 32037 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce1d7fc65f9146b5907a4cdee61776c4: Sending DeleteTablet for 3 replicas of tablet ea85a76efbd54ff987f6ca8ed01daa15
I20260806 02:14:57.913318 32319 tablet_service.cc:1514] Processing DeleteTablet for tablet ea85a76efbd54ff987f6ca8ed01daa15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:14:57 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:36422
I20260806 02:14:57.914433 32461 tablet_service.cc:1514] Processing DeleteTablet for tablet ea85a76efbd54ff987f6ca8ed01daa15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:14:57 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:49936
I20260806 02:14:57.914942 32171 tablet_service.cc:1514] Processing DeleteTablet for tablet ea85a76efbd54ff987f6ca8ed01daa15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:14:57 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:43614
I20260806 02:14:57.916185 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 32080
I20260806 02:14:57.921818 32587 tablet_replica.cc:330] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301: stopping tablet replica
I20260806 02:14:57.927922 32587 raft_consensus.cc:2229] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:14:57.928452 32587 raft_consensus.cc:2258] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:14:57.928747 32586 tablet_replica.cc:330] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996: stopping tablet replica
I20260806 02:14:57.929435 32586 raft_consensus.cc:2229] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:14:57.929797 32586 raft_consensus.cc:2258] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:14:57.931573 32586 ts_tablet_manager.cc:1905] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:14:57.936738 32586 ts_tablet_manager.cc:1918] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:14:57.936932 32586 log.cc:1196] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/ea85a76efbd54ff987f6ca8ed01daa15
I20260806 02:14:57.937393 32586 ts_tablet_manager.cc:1939] T ea85a76efbd54ff987f6ca8ed01daa15 P 706825a2ac8d46b99156e51aa592a996: Deleting consensus metadata
I20260806 02:14:57.938676 32587 ts_tablet_manager.cc:1905] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:14:57.938961 32025 catalog_manager.cc:4841] TS 706825a2ac8d46b99156e51aa592a996 (127.18.70.66:36851): tablet ea85a76efbd54ff987f6ca8ed01daa15 (table test-table [id=50db2b25e0394d5e96068333a3aba006]) successfully deleted
W20260806 02:14:57.940084 32024 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:42999 (error 108)
I20260806 02:14:57.940704 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 32247
I20260806 02:14:57.943475 32587 ts_tablet_manager.cc:1918] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:14:57.943596 32587 log.cc:1196] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/ea85a76efbd54ff987f6ca8ed01daa15
I20260806 02:14:57.943946 32587 ts_tablet_manager.cc:1939] T ea85a76efbd54ff987f6ca8ed01daa15 P d9653fecb9214d62aa4f808ae9104301: Deleting consensus metadata
I20260806 02:14:57.945168 32024 catalog_manager.cc:4841] TS d9653fecb9214d62aa4f808ae9104301 (127.18.70.67:35883): tablet ea85a76efbd54ff987f6ca8ed01daa15 (table test-table [id=50db2b25e0394d5e96068333a3aba006]) successfully deleted
I20260806 02:14:57.951149 32556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:14:57.950726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:42999 (local address 127.0.0.1:43640)
0806 02:14:57.950909 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 02:14:57.951021 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:42999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 02:14:57.951329 32024 catalog_manager.cc:4584] TS b2b39f23963745159da2d9ebd7e42f9f (127.18.70.65:42999): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea85a76efbd54ff987f6ca8ed01daa15: Network error: Client connection negotiation failed: client connection to 127.18.70.65:42999: connect: Connection refused (error 111)
I20260806 02:14:57.960243 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 32389
I20260806 02:14:57.977490 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 32005
I20260806 02:14:57.986604 18713 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 02:14:58.332016 18713 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 02:14:58.332121 18713 mini_ranger.cc:66] Stopping Ranger...
I20260806 02:14:58.677115 18713 mini_ranger.cc:68] Stopped Ranger
2026-08-06 02:14:58.677 UTC [31037] LOG: received smart shutdown request
2026-08-06 02:14:58.681 UTC [31037] LOG: background worker "logical replication launcher" (PID 31048) exited with exit code 1
2026-08-06 02:14:58.681 UTC [31042] LOG: shutting down
2026-08-06 02:14:58.840 UTC [31037] LOG: database system is shut down
2026-08-06T02:14:58Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47625 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:14:59 dist-test-slave-6k22 krb5kdc[32600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:14:59 dist-test-slave-6k22 krb5kdc[32600](info): set up 2 sockets
Aug 06 02:14:59 dist-test-slave-6k22 krb5kdc[32600](info): commencing operation
krb5kdc: starting...
W20260806 02:15:00.052851 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:15:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:15:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:15:00.178539 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:40693
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:40693,127.18.70.125:40771,127.18.70.124:44955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:00.289520 32615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:00.289774 32615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:00.289816 32615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:00.293736 32615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:15:00.293794 32615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:00.293813 32615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:00.293830 32615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:15:00.293845 32615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:15:00.298900 32615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:40693,127.18.70.125:40771,127.18.70.124:44955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:40693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:00.299904 32615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:00.300880 32615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:00.309237 32621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:00.309285 32624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:00.309262 32620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:00.309650 32615 server_base.cc:1034] running on GCE node
I20260806 02:15:00.310194 32615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:00.311259 32615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:00.312443 32615 hybrid_clock.cc:648] HybridClock initialized: now 1785982500312412 us; error 41 us; skew 500 ppm
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:00.315183 32615 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:15:00.316220 32615 webserver.cc:460] Webserver started at http://127.18.70.126:46409/ using document root <none> and password file <none>
I20260806 02:15:00.316766 32615 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:00.316838 32615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:00.317080 32615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:15:00.319090 32615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "5ae348a447844e289568a06d0f63f673"
format_stamp: "Formatted at 2026-08-06 02:15:00 on dist-test-slave-6k22"
server_key: "ff34dc8fff76418fde9bb4614541708c"
server_key_iv: "c2cbba5e242dee04bbaabac2334367ef"
server_key_version: "encryptionkey@0"
I20260806 02:15:00.319693 32615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "5ae348a447844e289568a06d0f63f673"
format_stamp: "Formatted at 2026-08-06 02:15:00 on dist-test-slave-6k22"
server_key: "ff34dc8fff76418fde9bb4614541708c"
server_key_iv: "c2cbba5e242dee04bbaabac2334367ef"
server_key_version: "encryptionkey@0"
I20260806 02:15:00.323328 32615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 02:15:00.325670 32631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:00.327013 32615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:15:00.327180 32615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "5ae348a447844e289568a06d0f63f673"
format_stamp: "Formatted at 2026-08-06 02:15:00 on dist-test-slave-6k22"
server_key: "ff34dc8fff76418fde9bb4614541708c"
server_key_iv: "c2cbba5e242dee04bbaabac2334367ef"
server_key_version: "encryptionkey@0"
I20260806 02:15:00.327299 32615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:00.343669 32615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:00.346540 32615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:00.346736 32615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:00.354142 32615 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:40693
I20260806 02:15:00.354167 32683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:40693 every 8 connection(s)
I20260806 02:15:00.355089 32615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:15:00.355485 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 32615
I20260806 02:15:00.355635 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:15:00.355911 18713 external_mini_cluster.cc:1428] Setting key d51ef6a5d55c6ba5f4b19e4b6f6b5aa6
I20260806 02:15:00.357914 32684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:00.361696 32684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.70.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.125@KRBTEST.COM" created.
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:00.377298 32688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.363924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.18.70.126:40693)
0806 02:15:00.364504 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.364517 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.364566 (+ 49us) server_negotiation.cc:407] Connection header received
0806 02:15:00.365618 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.365623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.365803 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.365989 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.366923 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.368011 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.369268 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.369887 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.370072 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.372485 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.372491 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.372501 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.372520 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.374104 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.374573 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.374584 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.374585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.374683 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.375008 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.375013 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.375015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.375383 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.375521 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.375921 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.376185 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":456,"spinlock_wait_cycles":23296,"thread_start_us":134,"threads_started":1}
I20260806 02:15:00.377312 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.363922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:40693 (local address 127.0.0.1:43888)
0806 02:15:00.364396 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.364445 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:00.365312 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:00.366012 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:00.366017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:00.366444 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:00.366737 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.366780 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.368136 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.368142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.369145 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.369155 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.370035 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.370038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.370223 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.370840 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:00.370861 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:00.372344 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:00.374220 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.374228 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.374238 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.374464 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.374799 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.374802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.374804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.374913 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.375492 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:00.375499 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:00.375679 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 02:15:00.375848 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 02:15:00.376236 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":141,"spinlock_wait_cycles":768,"thread_start_us":141,"threads_started":1}
I20260806 02:15:00.379856 32684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:15:00.380599 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.380358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40771 (local address 127.0.0.1:51118)
0806 02:15:00.380469 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.380541 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 02:15:00.380806 32633 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40771: connect: Connection refused (error 111)
I20260806 02:15:00.381676 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.381479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40771 (local address 127.0.0.1:51126)
0806 02:15:00.381576 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.381626 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 02:15:00.381819 32684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.125:40771: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40771: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.125.keytab.
Entry for principal kudu/127.18.70.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.125.keytab.
I20260806 02:15:00.382776 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.125:40771
--webserver_interface=127.18.70.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.125.keytab
--principal=kudu/127.18.70.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:40693,127.18.70.125:40771,127.18.70.124:44955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:15:00.431079 32684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } attempt: 1
I20260806 02:15:00.432037 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.431687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40771 (local address 127.0.0.1:51130)
0806 02:15:00.431850 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.431933 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 02:15:00.432889 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.432593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40771 (local address 127.0.0.1:51132)
0806 02:15:00.432749 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.432813 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 02:15:00.433102 32684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.125:40771: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40771: connect: Connection refused (error 111)
W20260806 02:15:00.496326 32691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:00.496582 32691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:00.496619 32691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:00.500583 32691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:15:00.500630 32691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:00.500649 32691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:00.500664 32691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:15:00.500679 32691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:15:00.501274 32684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } attempt: 2
I20260806 02:15:00.502120 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.501793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40771 (local address 127.0.0.1:51144)
0806 02:15:00.501963 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.502031 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 02:15:00.502808 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.502599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40771 (local address 127.0.0.1:51154)
0806 02:15:00.502698 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.502747 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 02:15:00.502933 32684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.125:40771: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40771: connect: Connection refused (error 111)
I20260806 02:15:00.506294 32691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:40693,127.18.70.125:40771,127.18.70.124:44955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.125.keytab
--principal=kudu/127.18.70.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.125:40771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.70.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:00.507218 32691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:00.508183 32691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:00.516395 32696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:00.516481 32691 server_base.cc:1034] running on GCE node
W20260806 02:15:00.516391 32700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:00.516391 32697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:00.517210 32691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:00.518374 32691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:00.519538 32691 hybrid_clock.cc:648] HybridClock initialized: now 1785982500519512 us; error 38 us; skew 500 ppm
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:00.522257 32691 init.cc:377] Logged in from keytab as kudu/127.18.70.125@KRBTEST.COM (short username kudu)
I20260806 02:15:00.523171 32691 webserver.cc:460] Webserver started at http://127.18.70.125:42703/ using document root <none> and password file <none>
I20260806 02:15:00.523794 32691 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:00.523892 32691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:00.524121 32691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:15:00.526120 32691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/data/instance:
uuid: "89dc127aa994465fb488a350aaa983a2"
format_stamp: "Formatted at 2026-08-06 02:15:00 on dist-test-slave-6k22"
server_key: "7f84e0f07099ac0c2ae79201627b6840"
server_key_iv: "656e827764712e564d56c2d7ed7c52fc"
server_key_version: "encryptionkey@0"
I20260806 02:15:00.526680 32691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/wal/instance:
uuid: "89dc127aa994465fb488a350aaa983a2"
format_stamp: "Formatted at 2026-08-06 02:15:00 on dist-test-slave-6k22"
server_key: "7f84e0f07099ac0c2ae79201627b6840"
server_key_iv: "656e827764712e564d56c2d7ed7c52fc"
server_key_version: "encryptionkey@0"
I20260806 02:15:00.530536 32691 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 02:15:00.533008 32707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:00.534163 32691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:15:00.534322 32691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/wal
uuid: "89dc127aa994465fb488a350aaa983a2"
format_stamp: "Formatted at 2026-08-06 02:15:00 on dist-test-slave-6k22"
server_key: "7f84e0f07099ac0c2ae79201627b6840"
server_key_iv: "656e827764712e564d56c2d7ed7c52fc"
server_key_version: "encryptionkey@0"
I20260806 02:15:00.534433 32691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:00.574317 32691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:00.577723 32691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:00.577930 32691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:00.585278 32691 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.125:40771
I20260806 02:15:00.585306 32763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.125:40771 every 8 connection(s)
I20260806 02:15:00.586212 32691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/data/info.pb
I20260806 02:15:00.588912 32764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:00.589686 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 32691
I20260806 02:15:00.589812 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-1/wal/instance
I20260806 02:15:00.590080 18713 external_mini_cluster.cc:1428] Setting key 55aecada5ab3862600cdb82b4851426a
I20260806 02:15:00.592592 32764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:15:00.594089 32684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } attempt: 3
WARNING: no policy specified for kudu/127.18.70.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.124@KRBTEST.COM" created.
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.125@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for kudu/127.18.70.125@KRBTEST.COM
I20260806 02:15:00.608736 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.594570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40771 (local address 127.0.0.1:51170)
0806 02:15:00.594709 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.594725 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:00.594837 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:00.597037 (+ 2200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:00.597039 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:00.597221 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:00.597341 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.597350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.599114 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.599118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.599997 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.600007 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.600581 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.600584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.600691 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.601195 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:00.601207 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:00.604084 (+ 2877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:00.606825 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.606828 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.606830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.607061 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.607487 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.607490 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.607492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.607598 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.608139 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:00.608142 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:00.608235 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 02:15:00.608369 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 02:15:00.608504 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":76}
I20260806 02:15:00.609441 32688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.594453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43898 (local address 127.18.70.126:40693)
0806 02:15:00.594615 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.594620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.595504 (+ 884us) server_negotiation.cc:407] Connection header received
0806 02:15:00.596715 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.596719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.596779 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.596884 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.597805 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.598419 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.599596 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.600145 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.600278 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.604238 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.604245 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.604251 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.604280 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.606058 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.606767 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.606774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.606779 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.606901 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.607300 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.607304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.607305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.607527 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.607666 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.609167 (+ 1501us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.609285 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":66,"spinlock_wait_cycles":1408}
I20260806 02:15:00.609790 300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.594856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.18.70.125:40771)
0806 02:15:00.595369 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.595387 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.595435 (+ 48us) server_negotiation.cc:407] Connection header received
0806 02:15:00.596520 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.596550 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.596869 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.597081 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.597521 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.598970 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.600113 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.600459 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.600566 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.604607 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.604614 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.604627 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.604657 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.606725 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.607272 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.607281 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.607286 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.607382 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.607750 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.607753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.607755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.608033 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.608169 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.609194 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.609329 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":403,"spinlock_wait_cycles":53120,"thread_start_us":138,"threads_started":1}
I20260806 02:15:00.609808 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.594851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:40693 (local address 127.0.0.1:43898)
0806 02:15:00.595366 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.595406 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:00.596518 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:00.596939 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:00.596948 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:00.597413 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:00.597675 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.597684 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.598526 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.598531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.599431 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.599442 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.600318 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.600322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.600483 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.601032 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:00.601058 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:00.603861 (+ 2803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:00.606188 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.606197 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.606206 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.606640 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.607013 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.607018 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.607022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.607190 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.607700 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:00.607709 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:00.608096 (+ 387us) client_negotiation.cc:769] Sending connection context
0806 02:15:00.608312 (+ 216us) client_negotiation.cc:240] Negotiation successful
0806 02:15:00.608671 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":27,"thread_start_us":138,"threads_started":1}
I20260806 02:15:00.611275 32764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:15:00.611761 32684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:15:00.612509 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.612204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53796)
0806 02:15:00.612340 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.612406 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 02:15:00.613292 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.613033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53802)
0806 02:15:00.613199 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.613241 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260806 02:15:00.613476 32684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:44955: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.125@KRBTEST.COM for kudu/127.18.70.125@KRBTEST.COM
I20260806 02:15:00.620141 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.611798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40771 (local address 127.0.0.1:51184)
0806 02:15:00.611939 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.611953 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:00.612022 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:00.612535 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:00.612538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:00.612695 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:00.612802 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.612809 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.614017 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.614020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.614548 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.614556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.615010 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.615013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.615098 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.615593 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:00.615608 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:00.617301 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:00.618771 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.618775 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.618777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.618953 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.619178 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.619180 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.619181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.619247 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.619684 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:00.619686 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:00.619796 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 02:15:00.619925 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 02:15:00.620019 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 02:15:00.620237 300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.612003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51184 (local address 127.18.70.125:40771)
0806 02:15:00.612359 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.612362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.612373 (+ 11us) server_negotiation.cc:407] Connection header received
0806 02:15:00.612398 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.612400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.612429 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.612526 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.613224 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.613868 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.614671 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.614908 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.615010 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.617422 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.617425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.617427 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.617448 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.618667 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.619026 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.619029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.619030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.619099 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.619358 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.619362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.619367 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.619518 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.619679 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.619979 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.620067 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298}
I20260806 02:15:00.620868 32764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:15:00.621482 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.621259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53814)
0806 02:15:00.621367 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.621435 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.124.keytab.
Entry for principal kudu/127.18.70.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.124.keytab.
W20260806 02:15:00.622104 32711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
I20260806 02:15:00.623003 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.622753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53826)
0806 02:15:00.622887 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.622942 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 02:15:00.622999 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.124:44955
--webserver_interface=127.18.70.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.124.keytab
--principal=kudu/127.18.70.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:40693,127.18.70.125:40771,127.18.70.124:44955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:00.623154 32764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:44955: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
I20260806 02:15:00.644666 32684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } attempt: 1
I20260806 02:15:00.645483 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.645156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53840)
0806 02:15:00.645322 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.645392 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 02:15:00.646203 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.645984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53856)
0806 02:15:00.646092 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.646143 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 02:15:00.646349 32684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:44955: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
I20260806 02:15:00.672425 32764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } attempt: 1
I20260806 02:15:00.673446 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.673141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53862)
0806 02:15:00.673296 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.673366 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 02:15:00.674233 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.674008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53866)
0806 02:15:00.674120 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.674173 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 02:15:00.674399 32764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:44955: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
I20260806 02:15:00.721546 32684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } attempt: 2
I20260806 02:15:00.722431 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.722107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53876)
0806 02:15:00.722274 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.722338 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 02:15:00.723248 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.723051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53888)
0806 02:15:00.723147 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.723191 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 02:15:00.723466 32684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:44955: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
W20260806 02:15:00.736452 303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:00.736670 303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:00.736708 303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:00.740738 303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:15:00.740787 303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:00.740804 303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:00.740820 303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:15:00.740864 303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:15:00.742563 32764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } attempt: 2
I20260806 02:15:00.743281 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.742997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53902)
0806 02:15:00.743150 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.743211 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 02:15:00.744016 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.743803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53910)
0806 02:15:00.743907 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.743960 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 02:15:00.744136 32764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:44955: Network error: Client connection negotiation failed: client connection to 127.18.70.124:44955: connect: Connection refused (error 111)
I20260806 02:15:00.746178 303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:40693,127.18.70.125:40771,127.18.70.124:44955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.124.keytab
--principal=kudu/127.18.70.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.124:44955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.70.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:00.747164 303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:00.748255 303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:00.756402 309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:00.756431 308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:00.756496 311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:00.756807 303 server_base.cc:1034] running on GCE node
I20260806 02:15:00.757355 303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:00.758458 303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:00.759625 303 hybrid_clock.cc:648] HybridClock initialized: now 1785982500759610 us; error 30 us; skew 500 ppm
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:00.762358 303 init.cc:377] Logged in from keytab as kudu/127.18.70.124@KRBTEST.COM (short username kudu)
I20260806 02:15:00.763280 303 webserver.cc:460] Webserver started at http://127.18.70.124:44073/ using document root <none> and password file <none>
I20260806 02:15:00.763907 303 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:00.763995 303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:00.764233 303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:15:00.766286 303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/data/instance:
uuid: "4bc8a3e734584979b2474c0200248c26"
format_stamp: "Formatted at 2026-08-06 02:15:00 on dist-test-slave-6k22"
server_key: "e7b07b05e44c2860641051058d186cd7"
server_key_iv: "aa9dcb21358def71baadbc926c25820e"
server_key_version: "encryptionkey@0"
I20260806 02:15:00.766866 303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/wal/instance:
uuid: "4bc8a3e734584979b2474c0200248c26"
format_stamp: "Formatted at 2026-08-06 02:15:00 on dist-test-slave-6k22"
server_key: "e7b07b05e44c2860641051058d186cd7"
server_key_iv: "aa9dcb21358def71baadbc926c25820e"
server_key_version: "encryptionkey@0"
I20260806 02:15:00.770484 303 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 02:15:00.772874 318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:00.774034 303 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:15:00.774166 303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/wal
uuid: "4bc8a3e734584979b2474c0200248c26"
format_stamp: "Formatted at 2026-08-06 02:15:00 on dist-test-slave-6k22"
server_key: "e7b07b05e44c2860641051058d186cd7"
server_key_iv: "aa9dcb21358def71baadbc926c25820e"
server_key_version: "encryptionkey@0"
I20260806 02:15:00.774273 303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:00.794772 303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:00.797542 303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:00.797731 303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:00.804530 303 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.124:44955
I20260806 02:15:00.804556 372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.124:44955 every 8 connection(s)
I20260806 02:15:00.805454 303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/data/info.pb
I20260806 02:15:00.808032 373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:00.809028 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 303
I20260806 02:15:00.809145 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785982339772911-18713-0/minicluster-data/master-2/wal/instance
I20260806 02:15:00.809413 18713 external_mini_cluster.cc:1428] Setting key cd9a512fce66024a4e3a7b2fa73246fd
I20260806 02:15:00.812136 373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.124@KRBTEST.COM
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.124@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:00.822626 32684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } attempt: 3
I20260806 02:15:00.822894 32688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.813773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43900 (local address 127.18.70.126:40693)
0806 02:15:00.813920 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.813924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.814169 (+ 245us) server_negotiation.cc:407] Connection header received
0806 02:15:00.814288 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.814294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.814349 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.814452 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.815149 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.815877 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.816921 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.817123 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.817222 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.819384 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.819389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.819395 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.819424 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.820940 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.821427 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.821432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.821433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.821516 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.821878 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.821884 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.821888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.822093 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.822190 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.822607 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.822733 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 02:15:00.823874 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.810690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53922 (local address 127.18.70.124:44955)
0806 02:15:00.811114 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.811129 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.811163 (+ 34us) server_negotiation.cc:407] Connection header received
0806 02:15:00.811845 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.811864 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.812170 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.812494 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.813312 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.814666 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.815732 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.816348 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.816499 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.818916 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.818921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.818931 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.818951 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.820889 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.821417 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.821422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.821424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.821552 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.821826 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.821828 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.821829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.822279 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.822432 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.822675 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.822895 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":135,"threads_started":1}
I20260806 02:15:00.823947 377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.813687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:40693 (local address 127.0.0.1:43900)
0806 02:15:00.814034 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.814081 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:00.814200 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:00.814513 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:00.814519 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:00.814919 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:00.815033 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.815039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.815978 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.815981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.816790 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.816798 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.817272 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.817276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.817368 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.817849 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:00.817864 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:00.819218 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:00.821041 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.821050 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.821063 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.821341 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.821629 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.821632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.821634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.821787 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.822201 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:00.822205 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:00.822340 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 02:15:00.822520 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 02:15:00.823780 (+ 1260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":97,"threads_started":1}
I20260806 02:15:00.825457 373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for kudu/127.18.70.124@KRBTEST.COM
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:00.832134 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.823056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53926)
0806 02:15:00.823245 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.823259 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:00.823340 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:00.823752 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:00.823755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:00.823922 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:00.824017 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.824025 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.825030 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.825034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.825799 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.825815 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.826368 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.826372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.826450 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.826941 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:00.826959 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:00.828333 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:00.830369 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.830372 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.830374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.830592 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.831076 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.831078 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.831080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.831158 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.831633 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:00.831637 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:00.831726 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 02:15:00.831844 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 02:15:00.831970 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260806 02:15:00.832239 378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.823163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53926 (local address 127.18.70.124:44955)
0806 02:15:00.823490 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.823499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.823513 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:15:00.823545 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.823547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.823611 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.824136 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.824220 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.824918 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.825937 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.826259 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.826368 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.828461 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.828464 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.828466 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.828488 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.830260 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.830698 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.830703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.830706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.830796 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.831255 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.831260 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.831261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.831484 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.831614 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.831914 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.832023 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":140,"threads_started":1}
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.124@KRBTEST.COM for kudu/127.18.70.125@KRBTEST.COM
I20260806 02:15:00.835289 32764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } attempt: 3
I20260806 02:15:00.835704 32684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673: Bootstrap starting.
I20260806 02:15:00.836644 32688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.826589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43902 (local address 127.18.70.126:40693)
0806 02:15:00.826678 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.826682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.826692 (+ 10us) server_negotiation.cc:407] Connection header received
0806 02:15:00.826726 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.826728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.826756 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.826840 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.827524 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.828037 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.829183 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.829550 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.829670 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.833177 (+ 3507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.833181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.833183 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.833206 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.834688 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.835138 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.835143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.835148 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.835227 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.835563 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.835566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.835568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.835813 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.835984 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.836357 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.836480 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 02:15:00.836997 300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.826030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51196 (local address 127.18.70.125:40771)
0806 02:15:00.826293 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.826298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.826322 (+ 24us) server_negotiation.cc:407] Connection header received
0806 02:15:00.826368 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.826371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.826421 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.826498 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.827375 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.828189 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.829904 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.830263 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.830357 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.833148 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.833152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.833154 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.833180 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.834939 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.835502 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.835507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.835509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.835641 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.836031 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.836034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.836037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.836255 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.836344 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.836694 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.836830 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260806 02:15:00.838081 377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.825919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40771 (local address 127.0.0.1:51196)
0806 02:15:00.826051 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.826062 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:00.826121 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:00.826920 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:00.826925 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:00.827095 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:00.827221 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.827230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.828310 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.828313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.829728 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.829739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.830530 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.830534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.830648 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.831096 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:00.831110 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:00.833006 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:00.835051 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.835056 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.835058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.835328 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.835765 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.835768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.835769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.835918 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.836428 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:00.836431 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:00.836516 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 02:15:00.837786 (+ 1270us) client_negotiation.cc:240] Negotiation successful
0806 02:15:00.837916 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 02:15:00.838681 32684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:00.839102 373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:15:00.839623 32684 log.cc:824] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673: Log is configured to *not* fsync() on all Append() calls
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.125@KRBTEST.COM for kudu/127.18.70.124@KRBTEST.COM
I20260806 02:15:00.843432 32684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673: No bootstrap required, opened a new log
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.125@KRBTEST.COM
Aug 06 02:15:00 dist-test-slave-6k22 krb5kdc[32600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.124@KRBTEST.COM for kudu/127.18.70.124@KRBTEST.COM
I20260806 02:15:00.846467 378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.835796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53940 (local address 127.18.70.124:44955)
0806 02:15:00.836490 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.836494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.836506 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:15:00.836536 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.836538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.836573 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.836804 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.837788 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.838777 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.840075 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.840457 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.840558 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.842610 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.842615 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.842619 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.842644 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.844319 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.844919 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.844924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.844929 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.845016 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.845354 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.845360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.845364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.845588 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.846057 (+ 469us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.846200 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.846308 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630}
I20260806 02:15:00.846943 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.835921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53940)
0806 02:15:00.836079 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.836093 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:00.836165 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:00.836691 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:00.836694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:00.837557 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:00.837669 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.837680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.839146 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.839149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.839923 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.839937 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.840570 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.840574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.840653 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.841126 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:00.841140 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:00.842473 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:00.844453 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.844459 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.844463 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.844729 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.845131 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.845134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.845136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.845226 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.845710 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:00.845714 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:00.845826 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 02:15:00.845918 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 02:15:00.846784 (+ 866us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260806 02:15:00.848239 32684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } }
I20260806 02:15:00.848477 32684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:00.848531 32684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae348a447844e289568a06d0f63f673, State: Initialized, Role: FOLLOWER
I20260806 02:15:00.849117 300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.838392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51206 (local address 127.18.70.125:40771)
0806 02:15:00.838622 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.838625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.838796 (+ 171us) server_negotiation.cc:407] Connection header received
0806 02:15:00.838955 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.838958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.838995 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.839077 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.840149 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.840692 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.841724 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.842113 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.842187 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.845006 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.845010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.845013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.845036 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.846637 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.847715 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.847719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.847720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.847802 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.848106 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.848109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.848111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.848331 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.848521 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.848876 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.848977 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260806 02:15:00.849059 32684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } }
I20260806 02:15:00.849846 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.839691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53942 (local address 127.18.70.124:44955)
0806 02:15:00.840090 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.840094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.840105 (+ 11us) server_negotiation.cc:407] Connection header received
0806 02:15:00.840131 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.840133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.840166 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.840231 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.840656 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.841403 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.842814 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.843766 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.843857 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.846538 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.846543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.846547 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.846575 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.848282 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.848743 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.848746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.848747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.848820 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.849058 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.849060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.849062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.849223 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.849301 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.849606 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.849717 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332}
I20260806 02:15:00.850231 377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.839523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:44955 (local address 127.0.0.1:53942)
0806 02:15:00.839917 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:00.839930 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:00.839997 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:00.840309 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:00.840312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:00.840465 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:00.840552 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.840558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.841525 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.841531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.842424 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:00.842437 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.843989 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.843992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:00.844082 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.844529 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:00.844547 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:00.846394 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:00.848393 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.848395 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.848397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.848639 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.848909 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:00.848911 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:00.848913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:00.848978 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:00.849363 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:00.849369 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:00.849452 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 02:15:00.849998 (+ 546us) client_negotiation.cc:240] Negotiation successful
0806 02:15:00.850110 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332}
I20260806 02:15:00.850507 32684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:15:00.850684 32764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2: Bootstrap starting.
I20260806 02:15:00.852699 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:15:00.852900 373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26: Bootstrap starting.
I20260806 02:15:00.853380 32764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:00.854228 32764 log.cc:824] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:00.855417 373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:00.856264 373 log.cc:824] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:00.857201 32764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2: No bootstrap required, opened a new log
W20260806 02:15:00.858165 392 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:15:00.858217 392 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:15:00.858198 373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26: No bootstrap required, opened a new log
I20260806 02:15:00.855394 32615 master_runner.cc:186] Error getting master registration for 127.18.70.125:40771: OK, instance_id { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" instance_seqno: 1785982500583196 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 02:15:00.860683 32615 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:15:00.861222 32764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } }
I20260806 02:15:00.861411 32764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:00.861464 32764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89dc127aa994465fb488a350aaa983a2, State: Initialized, Role: FOLLOWER
I20260806 02:15:00.862099 32764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } }
I20260806 02:15:00.863842 32764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:15:00.864225 373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } }
I20260806 02:15:00.864467 373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:00.864530 373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bc8a3e734584979b2474c0200248c26, State: Initialized, Role: FOLLOWER
I20260806 02:15:00.865010 373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } }
I20260806 02:15:00.866401 373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:15:00.868207 32688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.858634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43904 (local address 127.18.70.126:40693)
0806 02:15:00.858750 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.858755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.859098 (+ 343us) server_negotiation.cc:407] Connection header received
0806 02:15:00.859292 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.859296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.859354 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.859406 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.860430 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.861137 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.862263 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.862611 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.862691 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.864539 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.864543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.864545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.864570 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.866232 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.866762 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.866766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.866767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.866870 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.867182 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.867186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.867187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.867406 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.867518 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.867925 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.868040 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 02:15:00.869629 380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } } }
I20260806 02:15:00.869793 380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 02:15:00.871838 429 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:15:00.871909 429 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:15:00.876797 32691 master_runner.cc:263] No leader master found from master 89dc127aa994465fb488a350aaa983a2
I20260806 02:15:00.876816 303 master_runner.cc:263] No leader master found from master 4bc8a3e734584979b2474c0200248c26
I20260806 02:15:00.876868 32691 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:15:00.876873 303 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:15:00.878917 300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.859225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.18.70.125:40771)
0806 02:15:00.859327 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.859331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.867788 (+ 8457us) server_negotiation.cc:407] Connection header received
0806 02:15:00.867980 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.867983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.868030 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.868081 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.869952 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.870665 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.871785 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.872105 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.872172 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.874404 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.874409 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.874412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.874437 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.876416 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.877252 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.877256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.877259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.877358 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.877698 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.877702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.877704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.877920 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.878037 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.878663 (+ 626us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.878780 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 02:15:00.879895 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:00.859924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53956 (local address 127.18.70.124:44955)
0806 02:15:00.860033 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:00.860037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:00.868755 (+ 8718us) server_negotiation.cc:407] Connection header received
0806 02:15:00.869221 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:00.869224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:00.869266 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:00.869329 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:00.870191 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.870862 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.871880 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:00.872184 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:00.872260 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:00.875713 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:00.875717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:00.875720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:00.875749 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:00.877648 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.878369 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.878373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.878375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.878466 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:00.878783 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:00.878787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:00.878789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:00.879166 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:00.879274 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:00.879647 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 02:15:00.879758 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 02:15:00.883752 403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } } }
I20260806 02:15:00.883846 403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 02:15:00.884764 416 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:15:00.884835 416 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:15:00.891068 407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } } }
I20260806 02:15:00.891193 407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:15:00.916522 407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:15:00.916754 407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } }
I20260806 02:15:00.918409 32658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4bc8a3e734584979b2474c0200248c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae348a447844e289568a06d0f63f673" is_pre_election: true
I20260806 02:15:00.918780 32658 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bc8a3e734584979b2474c0200248c26 in term 0.
I20260806 02:15:00.919284 319 leader_election.cc:302] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bc8a3e734584979b2474c0200248c26, 5ae348a447844e289568a06d0f63f673; no voters:
I20260806 02:15:00.920395 32736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4bc8a3e734584979b2474c0200248c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89dc127aa994465fb488a350aaa983a2" is_pre_election: true
I20260806 02:15:00.920722 32736 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bc8a3e734584979b2474c0200248c26 in term 0.
I20260806 02:15:00.921133 407 leader_election.cc:288] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ae348a447844e289568a06d0f63f673 (127.18.70.126:40693), 89dc127aa994465fb488a350aaa983a2 (127.18.70.125:40771)
I20260806 02:15:00.921420 407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:15:00.921514 407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:15:00.921581 407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:00.923533 407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } }
I20260806 02:15:00.924326 32658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4bc8a3e734584979b2474c0200248c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae348a447844e289568a06d0f63f673"
I20260806 02:15:00.924476 32658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:00.925967 32658 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bc8a3e734584979b2474c0200248c26 in term 1.
I20260806 02:15:00.926337 319 leader_election.cc:302] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bc8a3e734584979b2474c0200248c26, 5ae348a447844e289568a06d0f63f673; no voters:
I20260806 02:15:00.929509 32736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4bc8a3e734584979b2474c0200248c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89dc127aa994465fb488a350aaa983a2"
I20260806 02:15:00.929672 32736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:00.931156 32736 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bc8a3e734584979b2474c0200248c26 in term 1.
I20260806 02:15:00.931627 407 leader_election.cc:288] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [CANDIDATE]: Term 1 election: Requested vote from peers 5ae348a447844e289568a06d0f63f673 (127.18.70.126:40693), 89dc127aa994465fb488a350aaa983a2 (127.18.70.125:40771)
I20260806 02:15:00.931828 407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:15:00.931927 407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [term 1 LEADER]: Becoming Leader. State: Replica: 4bc8a3e734584979b2474c0200248c26, State: Running, Role: LEADER
I20260806 02:15:00.932320 407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } }
I20260806 02:15:00.934366 407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bc8a3e734584979b2474c0200248c26. Latest consensus state: current_term: 1 leader_uuid: "4bc8a3e734584979b2474c0200248c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } } }
I20260806 02:15:00.934501 407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:00.935012 434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:15:00.936219 434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:15:00.946574 32736 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [term 1 FOLLOWER]: Refusing update from remote peer 4bc8a3e734584979b2474c0200248c26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:15:00.947316 432 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:15:00.951478 403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bc8a3e734584979b2474c0200248c26. Latest consensus state: current_term: 1 leader_uuid: "4bc8a3e734584979b2474c0200248c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } } }
I20260806 02:15:00.951670 403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:15:00.955453 32658 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [term 1 FOLLOWER]: Refusing update from remote peer 4bc8a3e734584979b2474c0200248c26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:15:00.956135 432 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:15:00.960381 380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bc8a3e734584979b2474c0200248c26. Latest consensus state: current_term: 1 leader_uuid: "4bc8a3e734584979b2474c0200248c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } } }
I20260806 02:15:00.960520 380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:15:00.970525 432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4bc8a3e734584979b2474c0200248c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } } }
I20260806 02:15:00.970709 432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:00.972178 403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4bc8a3e734584979b2474c0200248c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } } }
I20260806 02:15:00.972311 403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:15:00.972780 435 mvcc.cc:205] Tried to move back new op lower bound from 7315384323869458432 to 7315384323822628864. Current Snapshot: MvccSnapshot[applied={T|T < 7315384323869458432}]
I20260806 02:15:00.975737 407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4bc8a3e734584979b2474c0200248c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } } }
I20260806 02:15:00.975869 407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26 [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:00.977082 380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4bc8a3e734584979b2474c0200248c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ae348a447844e289568a06d0f63f673" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40693 } } peers { permanent_uuid: "89dc127aa994465fb488a350aaa983a2" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40771 } } peers { permanent_uuid: "4bc8a3e734584979b2474c0200248c26" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 44955 } } }
I20260806 02:15:00.977231 380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:15:00.978178 436 mvcc.cc:205] Tried to move back new op lower bound from 7315384323869458432 to 7315384323822628864. Current Snapshot: MvccSnapshot[applied={T|T < 7315384323869458432}]
I20260806 02:15:00.978401 438 mvcc.cc:205] Tried to move back new op lower bound from 7315384323869458432 to 7315384323822628864. Current Snapshot: MvccSnapshot[applied={T|T < 7315384323869458432}]
I20260806 02:15:00.987166 434 catalog_manager.cc:1347] Generated new cluster ID: e52d8bcb158b4fbb850d259d192f8457
I20260806 02:15:00.987239 434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:15:01.054639 434 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:15:01.055979 434 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:15:01.075095 434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bc8a3e734584979b2474c0200248c26: Generated new TSK 0
I20260806 02:15:01.076026 434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:15:01.859741 392 catalog_manager.cc:1259] Loaded cluster ID: e52d8bcb158b4fbb850d259d192f8457
I20260806 02:15:01.859831 392 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673: loading cluster ID for follower catalog manager: success
I20260806 02:15:01.861568 392 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673: acquiring CA information for follower catalog manager: success
I20260806 02:15:01.862860 392 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5ae348a447844e289568a06d0f63f673: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:15:01.886055 416 catalog_manager.cc:1259] Loaded cluster ID: e52d8bcb158b4fbb850d259d192f8457
I20260806 02:15:01.886139 416 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2: loading cluster ID for follower catalog manager: success
I20260806 02:15:01.887758 416 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2: acquiring CA information for follower catalog manager: success
I20260806 02:15:01.888854 416 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 89dc127aa994465fb488a350aaa983a2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:15:02.144959 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 32615
I20260806 02:15:02.153717 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 32691
I20260806 02:15:02.160822 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 303
2026-08-06T02:15:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3195 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:15:02 dist-test-slave-6k22 krb5kdc[448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:15:02 dist-test-slave-6k22 krb5kdc[448](info): set up 2 sockets
Aug 06 02:15:02 dist-test-slave-6k22 krb5kdc[448](info): commencing operation
krb5kdc: starting...
W20260806 02:15:03.216732 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:15:03 dist-test-slave-6k22 krb5kdc[448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:15:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:15:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:15:03.342875 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:33271
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:45121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:33271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:03.455533 461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:03.455776 461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:03.455848 461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:03.459714 461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:15:03.459761 461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:03.459805 461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:03.459843 461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:15:03.459867 461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:15:03.464813 461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:45121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:33271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:33271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:03.465873 461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:03.466889 461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 02:15:03.475617 461 server_base.cc:1034] running on GCE node
W20260806 02:15:03.475492 466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:03.475514 470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:03.475490 467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:03.476275 461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:03.477406 461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:03.478591 461 hybrid_clock.cc:648] HybridClock initialized: now 1785982503478571 us; error 30 us; skew 500 ppm
Aug 06 02:15:03 dist-test-slave-6k22 krb5kdc[448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982503, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:03.481383 461 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:15:03.482240 461 webserver.cc:460] Webserver started at http://127.18.70.126:43487/ using document root <none> and password file <none>
I20260806 02:15:03.482795 461 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:03.482883 461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:03.483114 461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:15:03.485414 461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "9c7f76d875384e96a209f68dfc6705a3"
format_stamp: "Formatted at 2026-08-06 02:15:03 on dist-test-slave-6k22"
server_key: "76f532d9cdbfbd8b7ec5d4669814cc9b"
server_key_iv: "fbc042d29135a13ab8c2e3b0242a04e4"
server_key_version: "encryptionkey@0"
I20260806 02:15:03.485996 461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "9c7f76d875384e96a209f68dfc6705a3"
format_stamp: "Formatted at 2026-08-06 02:15:03 on dist-test-slave-6k22"
server_key: "76f532d9cdbfbd8b7ec5d4669814cc9b"
server_key_iv: "fbc042d29135a13ab8c2e3b0242a04e4"
server_key_version: "encryptionkey@0"
I20260806 02:15:03.489794 461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 02:15:03.492238 477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:03.493429 461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 02:15:03.493567 461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "9c7f76d875384e96a209f68dfc6705a3"
format_stamp: "Formatted at 2026-08-06 02:15:03 on dist-test-slave-6k22"
server_key: "76f532d9cdbfbd8b7ec5d4669814cc9b"
server_key_iv: "fbc042d29135a13ab8c2e3b0242a04e4"
server_key_version: "encryptionkey@0"
I20260806 02:15:03.493678 461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:03.510854 461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:03.514120 461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:03.514322 461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:03.521485 461 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:33271
I20260806 02:15:03.521497 530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:33271 every 8 connection(s)
I20260806 02:15:03.522423 461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:15:03.525450 531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:03.529036 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 461
I20260806 02:15:03.529184 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:15:03.529452 18713 external_mini_cluster.cc:1428] Setting key 5cdf18f3e79597a154effe4cb23ee6b1
I20260806 02:15:03.531338 531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3: Bootstrap starting.
I20260806 02:15:03.533800 531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:03.534556 531 log.cc:824] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:03.536718 531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3: No bootstrap required, opened a new log
Aug 06 02:15:03 dist-test-slave-6k22 krb5kdc[448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:03.540001 531 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7f76d875384e96a209f68dfc6705a3" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33271 } }
I20260806 02:15:03.540206 531 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:03.540264 531 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7f76d875384e96a209f68dfc6705a3, State: Initialized, Role: FOLLOWER
I20260806 02:15:03.540747 531 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7f76d875384e96a209f68dfc6705a3" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33271 } }
I20260806 02:15:03.540877 531 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:15:03.540962 531 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:15:03.541090 531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:03.541891 531 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7f76d875384e96a209f68dfc6705a3" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33271 } }
I20260806 02:15:03.542222 531 leader_election.cc:302] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c7f76d875384e96a209f68dfc6705a3; no voters:
I20260806 02:15:03.542563 531 leader_election.cc:288] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:15:03.542632 536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:15:03.542906 536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [term 1 LEADER]: Becoming Leader. State: Replica: 9c7f76d875384e96a209f68dfc6705a3, State: Running, Role: LEADER
I20260806 02:15:03.543232 536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7f76d875384e96a209f68dfc6705a3" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33271 } }
I20260806 02:15:03.543857 531 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:15:03.544453 536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c7f76d875384e96a209f68dfc6705a3. Latest consensus state: current_term: 1 leader_uuid: "9c7f76d875384e96a209f68dfc6705a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7f76d875384e96a209f68dfc6705a3" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33271 } } }
I20260806 02:15:03.545132 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:03.530811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39812 (local address 127.18.70.126:33271)
0806 02:15:03.531217 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:03.531233 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:03.531269 (+ 36us) server_negotiation.cc:407] Connection header received
0806 02:15:03.531960 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:03.531978 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:03.532300 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:03.532574 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:03.533453 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:03.534522 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:03.535636 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:03.536081 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:03.536201 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:03.538948 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:03.538954 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:03.538967 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:03.538993 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:03.541417 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:03.541864 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:03.541877 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:03.541883 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:03.542032 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:03.542328 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:03.542331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:03.542333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:03.542863 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:03.543705 (+ 842us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:03.543827 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 02:15:03.544139 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":122,"threads_started":1}
I20260806 02:15:03.546684 536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:03.545135 537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c7f76d875384e96a209f68dfc6705a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7f76d875384e96a209f68dfc6705a3" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 33271 } } }
I20260806 02:15:03.547010 537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3 [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:03.548611 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 02:15:03.549407 550 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:15:03.549501 550 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:15:03.549583 552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:15:03.550304 552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:15:03.555497 552 catalog_manager.cc:1347] Generated new cluster ID: a0231544b1674c0f988f367d6adbda07
I20260806 02:15:03.555583 552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:15:03.593685 552 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:15:03.594713 552 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:15:03.599514 552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c7f76d875384e96a209f68dfc6705a3: Generated new TSK 0
I20260806 02:15:03.600211 552 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:15:03.604408 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 461
2026-08-06T02:15:03Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1441 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:15:03 dist-test-slave-6k22 krb5kdc[559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:15:03 dist-test-slave-6k22 krb5kdc[559](info): set up 2 sockets
Aug 06 02:15:03 dist-test-slave-6k22 krb5kdc[559](info): commencing operation
krb5kdc: starting...
W20260806 02:15:04.673617 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:15:04 dist-test-slave-6k22 krb5kdc[559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:15:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:15:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:15:04.798605 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:38937
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:38937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:04.910295 574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:04.910570 574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:04.910607 574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:04.914567 574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:15:04.914623 574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:04.914641 574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:04.914701 574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:15:04.914733 574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:15:04.919924 574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:38937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:38937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:04.920920 574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:04.921957 574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:04.930546 580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:04.930547 583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:04.930584 581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:04.930964 574 server_base.cc:1034] running on GCE node
I20260806 02:15:04.931478 574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:04.932571 574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:04.933756 574 hybrid_clock.cc:648] HybridClock initialized: now 1785982504933723 us; error 45 us; skew 500 ppm
Aug 06 02:15:04 dist-test-slave-6k22 krb5kdc[559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982504, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:04.941434 574 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:15:04.942319 574 webserver.cc:460] Webserver started at http://127.18.70.126:32859/ using document root <none> and password file <none>
I20260806 02:15:04.942827 574 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:04.942883 574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:04.943042 574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:15:04.945106 574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "6df0ca8b12504f7caee283811b0f92a7"
format_stamp: "Formatted at 2026-08-06 02:15:04 on dist-test-slave-6k22"
server_key: "cde4d3c984dd4bbf9c6c012d78fb277e"
server_key_iv: "a02144416e98256a0d0773df3cc992ad"
server_key_version: "encryptionkey@0"
I20260806 02:15:04.945640 574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "6df0ca8b12504f7caee283811b0f92a7"
format_stamp: "Formatted at 2026-08-06 02:15:04 on dist-test-slave-6k22"
server_key: "cde4d3c984dd4bbf9c6c012d78fb277e"
server_key_iv: "a02144416e98256a0d0773df3cc992ad"
server_key_version: "encryptionkey@0"
I20260806 02:15:04.949203 574 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:15:04.951543 591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:04.952821 574 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:15:04.952929 574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "6df0ca8b12504f7caee283811b0f92a7"
format_stamp: "Formatted at 2026-08-06 02:15:04 on dist-test-slave-6k22"
server_key: "cde4d3c984dd4bbf9c6c012d78fb277e"
server_key_iv: "a02144416e98256a0d0773df3cc992ad"
server_key_version: "encryptionkey@0"
I20260806 02:15:04.953011 574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:04.971149 574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:04.974380 574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:04.974555 574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:04.981769 574 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:38937
I20260806 02:15:04.982030 646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:38937 every 8 connection(s)
I20260806 02:15:04.982736 574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:15:04.984917 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 574
I20260806 02:15:04.985049 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:15:04.985381 18713 external_mini_cluster.cc:1428] Setting key e7cef9e3aef76195b6462b0752d10d54
I20260806 02:15:04.985895 647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:04.992831 647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Bootstrap starting.
Aug 06 02:15:04 dist-test-slave-6k22 krb5kdc[559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:04.995426 647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:04.996387 647 log.cc:824] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:04.998587 647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: No bootstrap required, opened a new log
I20260806 02:15:05.000247 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:04.986778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.18.70.126:38937)
0806 02:15:04.987197 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:04.987223 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:04.987259 (+ 36us) server_negotiation.cc:407] Connection header received
0806 02:15:04.987989 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:04.988008 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:04.988318 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:04.988608 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:04.989493 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:04.990432 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:04.991690 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:04.992136 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:04.992250 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:04.995045 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:04.995051 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:04.995064 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:04.995089 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:04.997092 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:04.997524 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:04.997530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:04.997535 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:04.997671 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:04.997953 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:04.997957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:04.997958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:04.998472 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:04.998659 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:04.998977 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 02:15:04.999283 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":161,"threads_started":1}
I20260806 02:15:05.002454 647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } }
I20260806 02:15:05.002661 647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:05.002697 647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df0ca8b12504f7caee283811b0f92a7, State: Initialized, Role: FOLLOWER
I20260806 02:15:05.003052 647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } }
I20260806 02:15:05.003172 647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:15:05.003219 647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:15:05.003283 647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:05.004158 647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } }
I20260806 02:15:05.004454 647 leader_election.cc:302] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6df0ca8b12504f7caee283811b0f92a7; no voters:
I20260806 02:15:05.004694 647 leader_election.cc:288] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:15:05.004820 652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:15:05.005084 652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 1 LEADER]: Becoming Leader. State: Replica: 6df0ca8b12504f7caee283811b0f92a7, State: Running, Role: LEADER
I20260806 02:15:05.005477 652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } }
I20260806 02:15:05.005937 647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:15:05.007090 653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6df0ca8b12504f7caee283811b0f92a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } } }
I20260806 02:15:05.007228 653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:05.007514 655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6df0ca8b12504f7caee283811b0f92a7. Latest consensus state: current_term: 1 leader_uuid: "6df0ca8b12504f7caee283811b0f92a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } } }
I20260806 02:15:05.007673 655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:05.007748 662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:15:05.010943 662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:15:05.016742 662 catalog_manager.cc:1347] Generated new cluster ID: 08212c9bf11e40678d3ba9c48aeceee7
I20260806 02:15:05.016834 662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:15:05.039260 662 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:15:05.040668 662 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:15:05.048771 662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Generated new TSK 0
I20260806 02:15:05.049646 662 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:15:05.087164 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:38937
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:05.201088 674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:05.201359 674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:05.201432 674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:05.205335 674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:05.205416 674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:05.205523 674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:15:05.210928 674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:05.211977 674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:05.212994 674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:05.222575 680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:05.222592 679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:05.222631 683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:05.222846 674 server_base.cc:1034] running on GCE node
I20260806 02:15:05.223270 674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:05.223934 674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:05.225098 674 hybrid_clock.cc:648] HybridClock initialized: now 1785982505225083 us; error 26 us; skew 500 ppm
Aug 06 02:15:05 dist-test-slave-6k22 krb5kdc[559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982505, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:05.231980 674 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:15:05.232813 674 webserver.cc:460] Webserver started at http://127.18.70.65:44611/ using document root <none> and password file <none>
I20260806 02:15:05.233336 674 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:05.233395 674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:05.233580 674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:15:05.235496 674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "f546d432630f400aa68ffd698cc36115"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "359ccfa6879a21be90bbac2be062d1ec"
server_key_iv: "79888f790f2428f97d1e7d373919cbe0"
server_key_version: "encryptionkey@0"
I20260806 02:15:05.236142 674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "f546d432630f400aa68ffd698cc36115"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "359ccfa6879a21be90bbac2be062d1ec"
server_key_iv: "79888f790f2428f97d1e7d373919cbe0"
server_key_version: "encryptionkey@0"
I20260806 02:15:05.239743 674 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:15:05.241938 690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:05.243070 674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:15:05.243212 674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "f546d432630f400aa68ffd698cc36115"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "359ccfa6879a21be90bbac2be062d1ec"
server_key_iv: "79888f790f2428f97d1e7d373919cbe0"
server_key_version: "encryptionkey@0"
I20260806 02:15:05.243317 674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:05.264354 674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:05.267513 674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:05.267709 674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:05.268303 674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:05.269053 674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:15:05.269098 674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:05.269135 674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:15:05.269156 674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:05.278133 674 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:35841
I20260806 02:15:05.278155 813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:35841 every 8 connection(s)
I20260806 02:15:05.279412 674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:15:05.284221 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 674
I20260806 02:15:05.284359 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:15:05.284675 18713 external_mini_cluster.cc:1428] Setting key 1fb6e58cadb00b94ba918601ca48fbc6
Aug 06 02:15:05 dist-test-slave-6k22 krb5kdc[559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982505, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:05.294705 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.281283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:42901 (local address 127.18.70.126:38937)
0806 02:15:05.281430 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:05.281434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:05.282196 (+ 762us) server_negotiation.cc:407] Connection header received
0806 02:15:05.283670 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:05.283674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:05.283732 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:05.283843 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:05.285450 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.286234 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.288045 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.288297 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.288425 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.290894 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:05.290898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:05.290901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:05.290933 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:05.292588 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:05.293142 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:05.293147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:05.293148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:05.293232 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:05.293595 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:05.293600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:05.293602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:05.293789 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:05.293903 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:05.294391 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 02:15:05.294521 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
I20260806 02:15:05.295622 816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.281613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:42901)
0806 02:15:05.282043 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:05.282087 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:05.283421 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:05.284044 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:05.284055 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:05.284589 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:05.285250 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.285273 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.286370 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.286373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.287923 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.287939 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.288470 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.288474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.288651 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.289250 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:05.289267 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:05.290719 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:05.292720 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:05.292726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:05.292741 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:05.293033 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:05.293336 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:05.293339 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:05.293341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:05.293492 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:05.293905 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:05.293914 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:05.294151 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 02:15:05.294339 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 02:15:05.294663 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":102,"threads_started":1}
I20260806 02:15:05.296962 814 heartbeater.cc:347] Connected to a master server at 127.18.70.126:38937
I20260806 02:15:05.297179 814 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
I20260806 02:15:05.297608 814 heartbeater.cc:511] Master 127.18.70.126:38937 requested a full tablet report, sending...
I20260806 02:15:05.299080 610 ts_manager.cc:194] Registered new tserver with Master: f546d432630f400aa68ffd698cc36115 (127.18.70.65:35841)
I20260806 02:15:05.300261 610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:42901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:15:05.314244 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:38937
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:05.442597 819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:05.442853 819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:05.442898 819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:05.446969 819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:05.447033 819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:05.447093 819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:15:05.452750 819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:05.454087 819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:05.455255 819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:05.465124 824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:05.465185 825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:05.465174 827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:05.465459 819 server_base.cc:1034] running on GCE node
I20260806 02:15:05.465827 819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:05.466449 819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:05.467669 819 hybrid_clock.cc:648] HybridClock initialized: now 1785982505467627 us; error 49 us; skew 500 ppm
Aug 06 02:15:05 dist-test-slave-6k22 krb5kdc[559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982505, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:05.470536 819 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:15:05.471429 819 webserver.cc:460] Webserver started at http://127.18.70.66:35783/ using document root <none> and password file <none>
I20260806 02:15:05.472028 819 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:05.472088 819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:05.472325 819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:15:05.474349 819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "a3520d2a11934d058ae5dbdf278a476c"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "bb71eccbca3bf9ab0dbbd70deb752ecf"
server_key_iv: "c322b5c92ecaefce2410b935d397787f"
server_key_version: "encryptionkey@0"
I20260806 02:15:05.474925 819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "a3520d2a11934d058ae5dbdf278a476c"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "bb71eccbca3bf9ab0dbbd70deb752ecf"
server_key_iv: "c322b5c92ecaefce2410b935d397787f"
server_key_version: "encryptionkey@0"
I20260806 02:15:05.478613 819 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 02:15:05.481062 834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:05.482354 819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:15:05.482491 819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "a3520d2a11934d058ae5dbdf278a476c"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "bb71eccbca3bf9ab0dbbd70deb752ecf"
server_key_iv: "c322b5c92ecaefce2410b935d397787f"
server_key_version: "encryptionkey@0"
I20260806 02:15:05.482599 819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:05.503916 819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:05.507000 819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:05.507202 819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:05.507967 819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:05.508846 819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:15:05.508917 819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:05.508989 819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:15:05.509038 819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:05.517976 819 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:44521
I20260806 02:15:05.518010 955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:44521 every 8 connection(s)
I20260806 02:15:05.518905 819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:15:05.520836 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 819
I20260806 02:15:05.520955 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:15:05.521220 18713 external_mini_cluster.cc:1428] Setting key 915bc6e1e011d3812791fd27c15f04e5
Aug 06 02:15:05 dist-test-slave-6k22 krb5kdc[559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982505, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
I20260806 02:15:05.535135 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.520728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:38667 (local address 127.18.70.126:38937)
0806 02:15:05.520917 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:05.520924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:05.521927 (+ 1003us) server_negotiation.cc:407] Connection header received
0806 02:15:05.523800 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:05.523803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:05.523844 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:05.523913 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:05.525425 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.526275 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.528204 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.528496 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.528581 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.531528 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:05.531531 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:05.531533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:05.531601 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:05.533138 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:05.533675 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:05.533681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:05.533685 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:05.533806 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:05.534156 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:05.534158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:05.534159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:05.534344 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:05.534422 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:05.534896 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 02:15:05.535012 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
I20260806 02:15:05.536126 958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.521208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:38667)
0806 02:15:05.521734 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:05.521803 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:05.523546 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:05.524094 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:05.524105 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:05.524658 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:05.525281 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.525296 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.526421 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.526428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.528071 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.528088 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.528673 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.528678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.528945 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.529604 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:05.529632 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:05.531339 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:05.533255 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:05.533261 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:05.533276 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:05.533567 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:05.533920 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:05.533923 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:05.533924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:05.534064 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:05.534436 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:05.534442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:05.534666 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 02:15:05.534823 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 02:15:05.535099 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":112,"threads_started":1}
I20260806 02:15:05.537554 956 heartbeater.cc:347] Connected to a master server at 127.18.70.126:38937
I20260806 02:15:05.537820 956 heartbeater.cc:464] Registering TS with master...
I20260806 02:15:05.538314 956 heartbeater.cc:511] Master 127.18.70.126:38937 requested a full tablet report, sending...
I20260806 02:15:05.539418 610 ts_manager.cc:194] Registered new tserver with Master: a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521)
I20260806 02:15:05.540037 610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:38667
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:15:05.550523 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:38937
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:05.670027 961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:05.670269 961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:05.670328 961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:05.674441 961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:05.674495 961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:05.674600 961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:15:05.679788 961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:05.680819 961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:05.681823 961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:05.691424 967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:05.691452 969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:05.691421 966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:05.691777 961 server_base.cc:1034] running on GCE node
I20260806 02:15:05.692111 961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:05.692736 961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:05.693933 961 hybrid_clock.cc:648] HybridClock initialized: now 1785982505693909 us; error 40 us; skew 500 ppm
Aug 06 02:15:05 dist-test-slave-6k22 krb5kdc[559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982505, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:05.696753 961 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:15:05.697592 961 webserver.cc:460] Webserver started at http://127.18.70.67:39181/ using document root <none> and password file <none>
I20260806 02:15:05.698143 961 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:05.698204 961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:05.698388 961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:15:05.700435 961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "1cb9f087a31040a68e69db6a93fc0398"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "7e7e4a230a59b1ac800825f241259704"
server_key_iv: "59a30b1631e2512fb9d9c3cc8c99e1e2"
server_key_version: "encryptionkey@0"
I20260806 02:15:05.700932 961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "1cb9f087a31040a68e69db6a93fc0398"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "7e7e4a230a59b1ac800825f241259704"
server_key_iv: "59a30b1631e2512fb9d9c3cc8c99e1e2"
server_key_version: "encryptionkey@0"
I20260806 02:15:05.704525 961 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:15:05.706938 977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:05.708278 961 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:15:05.708395 961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "1cb9f087a31040a68e69db6a93fc0398"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "7e7e4a230a59b1ac800825f241259704"
server_key_iv: "59a30b1631e2512fb9d9c3cc8c99e1e2"
server_key_version: "encryptionkey@0"
I20260806 02:15:05.708477 961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:05.724977 961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:05.728010 961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:05.728157 961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:05.728675 961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:05.729434 961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:15:05.729480 961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:05.729521 961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:15:05.729542 961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:05.738161 961 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:46671
I20260806 02:15:05.738190 1097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:46671 every 8 connection(s)
I20260806 02:15:05.739063 961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:15:05.747354 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 961
I20260806 02:15:05.747459 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:15:05.747835 18713 external_mini_cluster.cc:1428] Setting key 5454600920739b86aa220fd86b0fbd2e
Aug 06 02:15:05 dist-test-slave-6k22 krb5kdc[559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982505, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:05.752578 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.740855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:37675 (local address 127.18.70.126:38937)
0806 02:15:05.741007 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:05.741010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:05.741891 (+ 881us) server_negotiation.cc:407] Connection header received
0806 02:15:05.743231 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:05.743233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:05.743306 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:05.743395 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:05.744591 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.745110 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.746201 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.746409 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.746489 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.749237 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:05.749241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:05.749244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:05.749277 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:05.750750 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:05.751215 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:05.751218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:05.751219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:05.751290 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:05.751662 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:05.751665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:05.751666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:05.751839 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:05.751926 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:05.752369 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 02:15:05.752463 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 02:15:05.753315 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.741216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:37675)
0806 02:15:05.741720 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:05.741782 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:05.743012 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:05.743628 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:05.743635 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:05.743987 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:05.744464 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.744477 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.745216 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.745219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.746100 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.746109 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.746537 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.746540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.746695 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.747251 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:05.747267 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:05.749077 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:05.750856 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:05.750861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:05.750871 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:05.751116 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:05.751366 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:05.751368 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:05.751369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:05.751493 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:05.751940 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:05.751944 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:05.752152 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 02:15:05.752297 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 02:15:05.752521 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":105,"threads_started":1}
I20260806 02:15:05.754479 1098 heartbeater.cc:347] Connected to a master server at 127.18.70.126:38937
I20260806 02:15:05.754732 1098 heartbeater.cc:464] Registering TS with master...
I20260806 02:15:05.755198 1098 heartbeater.cc:511] Master 127.18.70.126:38937 requested a full tablet report, sending...
I20260806 02:15:05.756304 610 ts_manager.cc:194] Registered new tserver with Master: 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671)
I20260806 02:15:05.756850 610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:37675
I20260806 02:15:05.761615 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:15:05.771426 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.763593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46150 (local address 127.18.70.126:38937)
0806 02:15:05.763769 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:05.763774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:05.763960 (+ 186us) server_negotiation.cc:407] Connection header received
0806 02:15:05.764163 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:05.764173 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:05.764216 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:05.764315 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:05.765155 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.765700 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.766838 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.767062 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.767177 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.768572 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:05.768576 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:05.768578 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:05.768601 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:05.769701 (+ 1100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:05.770139 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:05.770143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:05.770145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:05.770224 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:05.770525 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:05.770529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:05.770530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:05.770732 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:05.770889 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:05.771137 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 02:15:05.771243 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 02:15:05.774235 610 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:15:05.777357 610 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:15:05.791939 1111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.785563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:44521 (local address 127.0.0.1:57376)
0806 02:15:05.785906 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:05.785959 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:05.786074 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:05.787172 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:05.787178 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:05.787217 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:05.787367 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.787374 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.788782 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.788786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.790195 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.790207 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.791299 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.791304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.791417 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.791455 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 02:15:05.791635 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 02:15:05.791762 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":27,"thread_start_us":78,"threads_started":1}
I20260806 02:15:05.791935 1114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.786246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57376 (local address 127.18.70.66:44521)
0806 02:15:05.786638 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:05.786651 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:05.786675 (+ 24us) server_negotiation.cc:407] Connection header received
0806 02:15:05.786763 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:05.786768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:05.786987 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:05.787182 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:05.787502 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.788633 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.790351 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.791160 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.791280 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.791439 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:05.791673 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 02:15:05.791769 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":181,"threads_started":1}
I20260806 02:15:05.792395 1113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.785919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41118 (local address 127.18.70.67:46671)
0806 02:15:05.787433 (+ 1514us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:05.787448 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:05.787474 (+ 26us) server_negotiation.cc:407] Connection header received
0806 02:15:05.787549 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:05.787597 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:05.787799 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:05.787950 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:05.788212 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.789147 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.790639 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.791194 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.791333 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.791479 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:05.792116 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 02:15:05.792256 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1396,"thread_start_us":158,"threads_started":1}
I20260806 02:15:05.792532 1116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.786933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48484 (local address 127.18.70.65:35841)
0806 02:15:05.787679 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:05.787693 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:05.787714 (+ 21us) server_negotiation.cc:407] Connection header received
0806 02:15:05.787785 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:05.787790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:05.787971 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:05.788104 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:05.788351 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.789343 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.790690 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.791481 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.791620 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.791762 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:05.792292 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 02:15:05.792377 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":105,"threads_started":1}
I20260806 02:15:05.792011 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.785737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:46671 (local address 127.0.0.1:41118)
0806 02:15:05.787247 (+ 1510us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:05.787262 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:05.787404 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:05.787927 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:05.787929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:05.787947 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:05.788076 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.788087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.789303 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.789306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.790517 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.790525 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.791529 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.791536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.791660 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.791682 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 02:15:05.791757 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 02:15:05.791833 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"mutex_wait_us":92,"thread_start_us":99,"threads_started":1}
I20260806 02:15:05.792196 1115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.786624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35841 (local address 127.0.0.1:48484)
0806 02:15:05.786969 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:05.786983 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:05.787077 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:05.788117 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:05.788122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:05.788144 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:05.788238 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.788245 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.789488 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.789491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.790537 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.790549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.791832 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.791835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.791926 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.791947 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 02:15:05.792015 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 02:15:05.792085 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":40,"thread_start_us":87,"threads_started":1}
I20260806 02:15:05.795094 887 tablet_service.cc:1467] Processing CreateTablet for tablet 32013e6b6d4940348ffc86fc9cf26457 (DEFAULT_TABLE table=test-table [id=2433bb4b31694ce38b5bb3466d52bb34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:15:05.796303 887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32013e6b6d4940348ffc86fc9cf26457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:05.798115 1026 tablet_service.cc:1467] Processing CreateTablet for tablet 32013e6b6d4940348ffc86fc9cf26457 (DEFAULT_TABLE table=test-table [id=2433bb4b31694ce38b5bb3466d52bb34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:15:05.798434 743 tablet_service.cc:1467] Processing CreateTablet for tablet 32013e6b6d4940348ffc86fc9cf26457 (DEFAULT_TABLE table=test-table [id=2433bb4b31694ce38b5bb3466d52bb34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:15:05.799031 1026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32013e6b6d4940348ffc86fc9cf26457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:05.799362 743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32013e6b6d4940348ffc86fc9cf26457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:05.804360 1117 tablet_bootstrap.cc:492] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: Bootstrap starting.
I20260806 02:15:05.805877 1118 tablet_bootstrap.cc:492] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398: Bootstrap starting.
I20260806 02:15:05.806311 1119 tablet_bootstrap.cc:492] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Bootstrap starting.
I20260806 02:15:05.808211 1118 tablet_bootstrap.cc:654] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:05.808776 1117 tablet_bootstrap.cc:654] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:05.809044 1119 tablet_bootstrap.cc:654] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:05.809470 1118 log.cc:824] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:05.809669 1117 log.cc:824] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:05.809757 1119 log.cc:824] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:05.811411 1118 tablet_bootstrap.cc:492] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398: No bootstrap required, opened a new log
I20260806 02:15:05.811652 1119 tablet_bootstrap.cc:492] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: No bootstrap required, opened a new log
I20260806 02:15:05.811652 1117 tablet_bootstrap.cc:492] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: No bootstrap required, opened a new log
I20260806 02:15:05.811661 1118 ts_tablet_manager.cc:1397] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:15:05.811836 1119 ts_tablet_manager.cc:1397] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:15:05.811856 1117 ts_tablet_manager.cc:1397] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 02:15:05.814968 1118 raft_consensus.cc:348] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:05.815207 1118 raft_consensus.cc:374] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:05.815267 1118 raft_consensus.cc:729] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb9f087a31040a68e69db6a93fc0398, State: Initialized, Role: FOLLOWER
I20260806 02:15:05.815712 1118 consensus_queue.cc:260] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:05.815862 1117 raft_consensus.cc:348] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:05.816133 1117 raft_consensus.cc:374] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:05.816190 1117 raft_consensus.cc:729] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3520d2a11934d058ae5dbdf278a476c, State: Initialized, Role: FOLLOWER
I20260806 02:15:05.816018 1119 raft_consensus.cc:348] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:05.816256 1119 raft_consensus.cc:374] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:05.816330 1119 raft_consensus.cc:729] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f546d432630f400aa68ffd698cc36115, State: Initialized, Role: FOLLOWER
I20260806 02:15:05.816591 1098 heartbeater.cc:503] Master 127.18.70.126:38937 was elected leader, sending a full tablet report...
I20260806 02:15:05.816716 1118 ts_tablet_manager.cc:1428] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 02:15:05.816653 1117 consensus_queue.cc:260] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:05.816701 1119 consensus_queue.cc:260] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:05.817466 956 heartbeater.cc:503] Master 127.18.70.126:38937 was elected leader, sending a full tablet report...
I20260806 02:15:05.817515 814 heartbeater.cc:503] Master 127.18.70.126:38937 was elected leader, sending a full tablet report...
I20260806 02:15:05.817585 1119 ts_tablet_manager.cc:1428] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:15:05.819039 1117 ts_tablet_manager.cc:1428] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 02:15:05.873145 1124 raft_consensus.cc:482] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:15:05.873400 1124 raft_consensus.cc:504] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:05.874670 1124 leader_election.cc:288] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521), f546d432630f400aa68ffd698cc36115 (127.18.70.65:35841)
I20260806 02:15:05.878744 1114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.875108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:49529 (local address 127.18.70.66:44521)
0806 02:15:05.875245 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:05.875251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:05.875265 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:15:05.875322 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:05.875325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:05.875375 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:05.875462 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:05.875901 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.876525 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.877639 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.878098 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.878191 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.878259 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:05.878564 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 02:15:05.878633 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 02:15:05.878752 1116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.875108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:57425 (local address 127.18.70.65:35841)
0806 02:15:05.875239 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:05.875246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:05.875517 (+ 271us) server_negotiation.cc:407] Connection header received
0806 02:15:05.875698 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:05.875702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:05.875773 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:05.875869 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:05.876158 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.876677 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.877768 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.878172 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.878288 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.878342 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:05.878538 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 02:15:05.878606 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:15:05.878750 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.874870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:44521 (local address 127.18.70.67:49529)
0806 02:15:05.875004 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:05.875021 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:05.875133 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:05.875503 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:05.875508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:05.875548 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:05.875739 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.875747 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.876630 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.876634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.877513 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.877523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.878235 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.878242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.878350 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.878442 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 02:15:05.878533 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 02:15:05.878606 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":61,"spinlock_wait_cycles":5760}
I20260806 02:15:05.878770 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.874965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35841 (local address 127.18.70.67:57425)
0806 02:15:05.875421 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:05.875437 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:05.875542 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:05.875894 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:05.875896 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:05.875914 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:05.876033 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.876039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.876806 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.876808 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.877641 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:05.877652 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.878308 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.878311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:05.878417 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.878439 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 02:15:05.878499 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 02:15:05.878589 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":22,"spinlock_wait_cycles":4352,"thread_start_us":139,"threads_started":1}
I20260806 02:15:05.879539 907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32013e6b6d4940348ffc86fc9cf26457" candidate_uuid: "1cb9f087a31040a68e69db6a93fc0398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3520d2a11934d058ae5dbdf278a476c" is_pre_election: true
I20260806 02:15:05.879664 766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32013e6b6d4940348ffc86fc9cf26457" candidate_uuid: "1cb9f087a31040a68e69db6a93fc0398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f546d432630f400aa68ffd698cc36115" is_pre_election: true
I20260806 02:15:05.879920 907 raft_consensus.cc:2454] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cb9f087a31040a68e69db6a93fc0398 in term 0.
I20260806 02:15:05.879959 766 raft_consensus.cc:2454] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cb9f087a31040a68e69db6a93fc0398 in term 0.
I20260806 02:15:05.880403 979 leader_election.cc:302] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1cb9f087a31040a68e69db6a93fc0398, a3520d2a11934d058ae5dbdf278a476c, f546d432630f400aa68ffd698cc36115; no voters:
I20260806 02:15:05.880663 1124 raft_consensus.cc:2781] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:15:05.880750 1124 raft_consensus.cc:482] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:15:05.880803 1124 raft_consensus.cc:3037] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:05.881808 1124 raft_consensus.cc:504] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:05.882139 1124 leader_election.cc:288] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [CANDIDATE]: Term 1 election: Requested vote from peers a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521), f546d432630f400aa68ffd698cc36115 (127.18.70.65:35841)
I20260806 02:15:05.882433 907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32013e6b6d4940348ffc86fc9cf26457" candidate_uuid: "1cb9f087a31040a68e69db6a93fc0398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3520d2a11934d058ae5dbdf278a476c"
I20260806 02:15:05.882539 766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32013e6b6d4940348ffc86fc9cf26457" candidate_uuid: "1cb9f087a31040a68e69db6a93fc0398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f546d432630f400aa68ffd698cc36115"
I20260806 02:15:05.882570 907 raft_consensus.cc:3037] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:05.882658 766 raft_consensus.cc:3037] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:05.883960 766 raft_consensus.cc:2454] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb9f087a31040a68e69db6a93fc0398 in term 1.
I20260806 02:15:05.883960 907 raft_consensus.cc:2454] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cb9f087a31040a68e69db6a93fc0398 in term 1.
I20260806 02:15:05.884241 979 leader_election.cc:302] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb9f087a31040a68e69db6a93fc0398, a3520d2a11934d058ae5dbdf278a476c; no voters:
I20260806 02:15:05.884492 1124 raft_consensus.cc:2781] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:15:05.884748 1124 raft_consensus.cc:686] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [term 1 LEADER]: Becoming Leader. State: Replica: 1cb9f087a31040a68e69db6a93fc0398, State: Running, Role: LEADER
I20260806 02:15:05.885094 1124 consensus_queue.cc:237] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:05.888387 608 catalog_manager.cc:5495] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67). New cstate: current_term: 1 leader_uuid: "1cb9f087a31040a68e69db6a93fc0398" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } health_report { overall_health: UNKNOWN } } }
I20260806 02:15:05.912544 1113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.908967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41124 (local address 127.18.70.67:46671)
0806 02:15:05.909093 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:05.909097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:05.909118 (+ 21us) server_negotiation.cc:407] Connection header received
0806 02:15:05.909243 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:05.909245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:05.909294 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:05.909384 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:15:05.909614 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.910120 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.910929 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:05.911290 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:05.911378 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:05.911693 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:15:05.912125 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:15:05.912248 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:05.912359 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 02:15:05.912429 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 02:15:05.918406 766 raft_consensus.cc:1261] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Refusing update from remote peer 1cb9f087a31040a68e69db6a93fc0398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:15:05.918408 907 raft_consensus.cc:1261] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c [term 1 FOLLOWER]: Refusing update from remote peer 1cb9f087a31040a68e69db6a93fc0398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:15:05.919135 1124 consensus_queue.cc:1035] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:15:05.919327 1128 consensus_queue.cc:1035] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:15:05.927032 1132 mvcc.cc:205] Tried to move back new op lower bound from 7315384344234844160 to 7315384344108417024. Current Snapshot: MvccSnapshot[applied={T|T < 7315384344234844160}]
I20260806 02:15:05.927189 1131 mvcc.cc:205] Tried to move back new op lower bound from 7315384344234844160 to 7315384344108417024. Current Snapshot: MvccSnapshot[applied={T|T < 7315384344234844160}]
I20260806 02:15:05.935262 609 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:15:05.935425 609 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:15:05.937675 609 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Sending DeleteTablet for 3 replicas of tablet 32013e6b6d4940348ffc86fc9cf26457
I20260806 02:15:05.938390 887 tablet_service.cc:1514] Processing DeleteTablet for tablet 32013e6b6d4940348ffc86fc9cf26457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:15:05 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:57376
I20260806 02:15:05.938397 743 tablet_service.cc:1514] Processing DeleteTablet for tablet 32013e6b6d4940348ffc86fc9cf26457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:15:05 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:48484
I20260806 02:15:05.938525 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 674
I20260806 02:15:05.938468 1026 tablet_service.cc:1514] Processing DeleteTablet for tablet 32013e6b6d4940348ffc86fc9cf26457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:15:05 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:41118
I20260806 02:15:05.939255 1143 tablet_replica.cc:330] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398: stopping tablet replica
I20260806 02:15:05.939580 1143 raft_consensus.cc:2229] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:15:05.939904 1143 raft_consensus.cc:2258] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:15:05.943786 1142 tablet_replica.cc:330] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: stopping tablet replica
I20260806 02:15:05.944123 1142 raft_consensus.cc:2229] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:15:05.944455 1142 raft_consensus.cc:2258] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:15:05.945075 1143 ts_tablet_manager.cc:1905] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:15:05.946051 1142 ts_tablet_manager.cc:1905] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:15:05.948451 1143 ts_tablet_manager.cc:1918] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:15:05.948602 1143 log.cc:1196] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/32013e6b6d4940348ffc86fc9cf26457
W20260806 02:15:05.948776 595 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:35841 (error 108)
I20260806 02:15:05.949016 1143 ts_tablet_manager.cc:1939] T 32013e6b6d4940348ffc86fc9cf26457 P 1cb9f087a31040a68e69db6a93fc0398: Deleting consensus metadata
I20260806 02:15:05.949580 1142 ts_tablet_manager.cc:1918] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:15:05.949757 1142 log.cc:1196] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/32013e6b6d4940348ffc86fc9cf26457
I20260806 02:15:05.950138 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 819
I20260806 02:15:05.950177 1115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.949897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35841 (local address 127.0.0.1:48494)
0806 02:15:05.950013 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:05.950087 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:35841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 02:15:05.950977 594 catalog_manager.cc:4841] TS 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671): tablet 32013e6b6d4940348ffc86fc9cf26457 (table test-table [id=2433bb4b31694ce38b5bb3466d52bb34]) successfully deleted
W20260806 02:15:05.952286 595 catalog_manager.cc:4584] TS f546d432630f400aa68ffd698cc36115 (127.18.70.65:35841): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 32013e6b6d4940348ffc86fc9cf26457: Network error: Client connection negotiation failed: client connection to 127.18.70.65:35841: connect: Connection refused (error 111)
I20260806 02:15:05.960300 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 961
I20260806 02:15:05.961549 1115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:05.960471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:44521 (local address 127.0.0.1:57386)
0806 02:15:05.961388 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:05.961461 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.66:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":828}
I20260806 02:15:05.968292 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 574
I20260806 02:15:05.975822 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:38937
--webserver_interface=127.18.70.126
--webserver_port=32859
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:38937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:06.088276 1145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:06.088579 1145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:06.088617 1145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:06.092556 1145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:15:06.092609 1145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:06.092628 1145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:06.092643 1145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:15:06.092689 1145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:15:06.097982 1145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:38937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:38937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=32859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:06.099054 1145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:06.100164 1145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:06.114164 1152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:06.114338 1145 server_base.cc:1034] running on GCE node
W20260806 02:15:06.114153 1151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:06.114148 1154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:06.115123 1145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:06.116330 1145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:06.117532 1145 hybrid_clock.cc:648] HybridClock initialized: now 1785982506117512 us; error 30 us; skew 500 ppm
Aug 06 02:15:06 dist-test-slave-6k22 krb5kdc[559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982506, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:06.120293 1145 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:15:06.121260 1145 webserver.cc:460] Webserver started at http://127.18.70.126:32859/ using document root <none> and password file <none>
I20260806 02:15:06.121917 1145 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:06.122057 1145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:06.126000 1145 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260806 02:15:06.128331 1161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:06.129563 1145 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:15:06.129699 1145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "6df0ca8b12504f7caee283811b0f92a7"
format_stamp: "Formatted at 2026-08-06 02:15:04 on dist-test-slave-6k22"
server_key: "cde4d3c984dd4bbf9c6c012d78fb277e"
server_key_iv: "a02144416e98256a0d0773df3cc992ad"
server_key_version: "encryptionkey@0"
I20260806 02:15:06.130304 1145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:06.160041 1145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:06.163389 1145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:06.163651 1145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:06.171123 1145 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:38937
I20260806 02:15:06.171437 1217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:38937 every 8 connection(s)
I20260806 02:15:06.172276 1145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:15:06.173306 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 1145
I20260806 02:15:06.173899 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:35841
--local_ip_for_outbound_sockets=127.18.70.65
--tserver_master_addrs=127.18.70.126:38937
--webserver_port=44611
--webserver_interface=127.18.70.65
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:15:06.178860 1218 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:15:06.180794 1218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Bootstrap starting.
I20260806 02:15:06.198282 1218 log.cc:824] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:06.206727 1218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:15:06.207098 1218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Bootstrap complete.
I20260806 02:15:06.211139 1218 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } }
I20260806 02:15:06.211556 1218 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df0ca8b12504f7caee283811b0f92a7, State: Initialized, Role: FOLLOWER
I20260806 02:15:06.212173 1218 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } }
I20260806 02:15:06.212323 1218 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:15:06.212422 1218 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:15:06.212524 1218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 1 FOLLOWER]: Advancing to term 2
I20260806 02:15:06.214354 1218 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } }
I20260806 02:15:06.214695 1218 leader_election.cc:302] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6df0ca8b12504f7caee283811b0f92a7; no voters:
I20260806 02:15:06.215034 1218 leader_election.cc:288] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 02:15:06.215117 1223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 02:15:06.215355 1223 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 2 LEADER]: Becoming Leader. State: Replica: 6df0ca8b12504f7caee283811b0f92a7, State: Running, Role: LEADER
I20260806 02:15:06.215673 1223 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } }
I20260806 02:15:06.216336 1218 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:15:06.217149 1225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6df0ca8b12504f7caee283811b0f92a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } } }
I20260806 02:15:06.217188 1224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6df0ca8b12504f7caee283811b0f92a7. Latest consensus state: current_term: 2 leader_uuid: "6df0ca8b12504f7caee283811b0f92a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } } }
I20260806 02:15:06.217298 1225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:06.217304 1224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:06.217715 1229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:15:06.222604 1229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:15:06.223425 1229 catalog_manager.cc:1259] Loaded cluster ID: 08212c9bf11e40678d3ba9c48aeceee7
I20260806 02:15:06.223503 1229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:15:06.224632 1229 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 02:15:06.227463 1229 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 02:15:06.231827 1229 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:15:06.232447 1229 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Loaded TSK: 0
I20260806 02:15:06.233515 1229 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 02:15:06.308586 1221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:06.308972 1221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:06.309053 1221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:06.315070 1221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:06.315158 1221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:06.315255 1221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:15:06.321024 1221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:35841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=44611
--tserver_master_addrs=127.18.70.126:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:06.322103 1221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:06.323221 1221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:06.334161 1248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:06.334197 1247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:06.334221 1221 server_base.cc:1034] running on GCE node
W20260806 02:15:06.334589 1250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:06.335088 1221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:06.335919 1221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:06.337124 1221 hybrid_clock.cc:648] HybridClock initialized: now 1785982506337100 us; error 35 us; skew 500 ppm
Aug 06 02:15:06 dist-test-slave-6k22 krb5kdc[559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982506, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:06.340710 1221 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:15:06.341817 1221 webserver.cc:460] Webserver started at http://127.18.70.65:44611/ using document root <none> and password file <none>
I20260806 02:15:06.342592 1221 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:06.342702 1221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:06.347944 1221 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:15:06.350653 1257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:06.352312 1221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:15:06.352444 1221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "f546d432630f400aa68ffd698cc36115"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "359ccfa6879a21be90bbac2be062d1ec"
server_key_iv: "79888f790f2428f97d1e7d373919cbe0"
server_key_version: "encryptionkey@0"
I20260806 02:15:06.353152 1221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:06.372676 1221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:06.375936 1221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:06.376139 1221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:06.376804 1221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:06.377979 1264 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 02:15:06.382424 1221 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 02:15:06.382501 1221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260806 02:15:06.382568 1221 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 02:15:06.384402 1221 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 02:15:06.384462 1221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 02:15:06.384738 1264 tablet_bootstrap.cc:492] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Bootstrap starting.
I20260806 02:15:06.395059 1221 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:35841
I20260806 02:15:06.395107 1382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:35841 every 8 connection(s)
I20260806 02:15:06.396451 1221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:15:06.404203 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 1221
I20260806 02:15:06.404680 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:44521
--local_ip_for_outbound_sockets=127.18.70.66
--tserver_master_addrs=127.18.70.126:38937
--webserver_port=35783
--webserver_interface=127.18.70.66
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:15:06.406437 1264 log.cc:824] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Log is configured to *not* fsync() on all Append() calls
Aug 06 02:15:06 dist-test-slave-6k22 krb5kdc[559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982506, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:06.415019 1264 tablet_bootstrap.cc:492] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:15:06.415472 1264 tablet_bootstrap.cc:492] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Bootstrap complete.
I20260806 02:15:06.415887 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.398642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:52403 (local address 127.18.70.126:38937)
0806 02:15:06.399000 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:06.399017 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:06.399506 (+ 489us) server_negotiation.cc:407] Connection header received
0806 02:15:06.400655 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:06.400666 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:06.400974 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:06.401302 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:06.402443 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.403442 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.405111 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.405796 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.406004 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.409163 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:06.409172 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:06.409192 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:06.409231 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:06.411681 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:06.412342 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:06.412350 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:06.412357 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:06.412531 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:06.412952 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:06.412956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:06.412958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:06.413393 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:06.413534 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:06.414272 (+ 738us) server_negotiation.cc:299] Negotiation successful
0806 02:15:06.414616 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":123,"threads_started":1}
I20260806 02:15:06.416062 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.398767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:52403)
0806 02:15:06.399333 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:06.399376 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:06.400239 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:06.401373 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:06.401381 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:06.401804 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:06.402249 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.402262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.403639 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.403648 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.404968 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.404983 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.405980 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.405986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.406222 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.407015 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:06.407040 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:06.408947 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:06.411809 (+ 2862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:06.411815 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:06.411828 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:06.412165 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:06.412655 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:06.412658 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:06.412660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:06.412825 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:06.413607 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:06.413613 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:06.413877 (+ 264us) client_negotiation.cc:769] Sending connection context
0806 02:15:06.414063 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 02:15:06.414382 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":172,"threads_started":1}
I20260806 02:15:06.417018 1264 ts_tablet_manager.cc:1397] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Time spent bootstrapping tablet: real 0.033s user 0.019s sys 0.012s
I20260806 02:15:06.419298 1383 heartbeater.cc:347] Connected to a master server at 127.18.70.126:38937
I20260806 02:15:06.419534 1383 heartbeater.cc:464] Registering TS with master...
I20260806 02:15:06.420270 1383 heartbeater.cc:511] Master 127.18.70.126:38937 requested a full tablet report, sending...
I20260806 02:15:06.419776 1264 raft_consensus.cc:348] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:06.420432 1264 raft_consensus.cc:729] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f546d432630f400aa68ffd698cc36115, State: Initialized, Role: FOLLOWER
I20260806 02:15:06.420905 1264 consensus_queue.cc:260] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:06.421849 1264 ts_tablet_manager.cc:1428] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260806 02:15:06.423640 1178 ts_manager.cc:194] Registered new tserver with Master: f546d432630f400aa68ffd698cc36115 (127.18.70.65:35841)
I20260806 02:15:06.427413 1178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:52403
I20260806 02:15:06.429726 1383 heartbeater.cc:503] Master 127.18.70.126:38937 was elected leader, sending a full tablet report...
Aug 06 02:15:06 dist-test-slave-6k22 krb5kdc[559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982506, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for kudu/127.18.70.65@KRBTEST.COM
I20260806 02:15:06.437712 1390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.426875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35841 (local address 127.0.0.1:48498)
0806 02:15:06.427544 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:06.427636 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:06.427798 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:06.428274 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:06.428279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:06.428674 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:06.428795 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.428802 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.430348 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.430352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.431289 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.431299 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.431979 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.431983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.432096 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.432676 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:06.432689 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:06.434129 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:06.436074 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:06.436084 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:06.436090 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:06.436301 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:06.436594 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:06.436597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:06.436598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:06.436686 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:06.437094 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:06.437100 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:06.437216 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 02:15:06.437343 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 02:15:06.437517 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":93,"threads_started":1}
I20260806 02:15:06.437739 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.426918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48498 (local address 127.18.70.65:35841)
0806 02:15:06.427277 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:06.427287 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:06.427735 (+ 448us) server_negotiation.cc:407] Connection header received
0806 02:15:06.427957 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:06.427961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:06.428110 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:06.428283 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:06.428985 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.430217 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.431406 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.431856 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.431977 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.434267 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:06.434277 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:06.434289 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:06.434315 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:06.435965 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:06.436413 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:06.436418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:06.436420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:06.436497 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:06.436771 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:06.436774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:06.436776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:06.436983 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:06.437118 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:06.437430 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 02:15:06.437600 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":96,"threads_started":1}
I20260806 02:15:06.439064 1308 tablet_service.cc:1514] Processing DeleteTablet for tablet 32013e6b6d4940348ffc86fc9cf26457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:15:05 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:48498
I20260806 02:15:06.440044 1392 tablet_replica.cc:330] stopping tablet replica
I20260806 02:15:06.440356 1392 raft_consensus.cc:2229] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:15:06.440626 1392 raft_consensus.cc:2258] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:15:06.441560 1392 ts_tablet_manager.cc:1905] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:15:06.444964 1392 ts_tablet_manager.cc:1918] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:15:06.445101 1392 log.cc:1196] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal/wals/32013e6b6d4940348ffc86fc9cf26457
I20260806 02:15:06.445387 1392 ts_tablet_manager.cc:1939] T 32013e6b6d4940348ffc86fc9cf26457 P f546d432630f400aa68ffd698cc36115: Deleting consensus metadata
I20260806 02:15:06.447041 1165 catalog_manager.cc:4841] TS f546d432630f400aa68ffd698cc36115 (127.18.70.65:35841): tablet 32013e6b6d4940348ffc86fc9cf26457 (table test-table [id=2433bb4b31694ce38b5bb3466d52bb34]) successfully deleted
W20260806 02:15:06.527338 1387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:06.527626 1387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:06.527669 1387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:06.531965 1387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:06.532019 1387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:06.532078 1387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:15:06.537485 1387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:44521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=35783
--tserver_master_addrs=127.18.70.126:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:06.538522 1387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:06.539477 1387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:06.549382 1401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:06.549379 1399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:06.549401 1398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:06.549780 1387 server_base.cc:1034] running on GCE node
I20260806 02:15:06.550239 1387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:06.550844 1387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:06.551999 1387 hybrid_clock.cc:648] HybridClock initialized: now 1785982506551967 us; error 42 us; skew 500 ppm
Aug 06 02:15:06 dist-test-slave-6k22 krb5kdc[559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982506, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:06.554774 1387 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:15:06.555665 1387 webserver.cc:460] Webserver started at http://127.18.70.66:35783/ using document root <none> and password file <none>
I20260806 02:15:06.556244 1387 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:06.556341 1387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:06.560217 1387 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:15:06.562491 1411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:06.563812 1387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:15:06.563927 1387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "a3520d2a11934d058ae5dbdf278a476c"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "bb71eccbca3bf9ab0dbbd70deb752ecf"
server_key_iv: "c322b5c92ecaefce2410b935d397787f"
server_key_version: "encryptionkey@0"
I20260806 02:15:06.564442 1387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:06.602541 1387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:06.606070 1387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:06.606230 1387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:06.606889 1387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:06.608016 1419 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260806 02:15:06.611919 1419 ts_tablet_manager.cc:1852] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260806 02:15:06.612396 1419 ts_tablet_manager.cc:1905] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:15:06.616472 1419 ts_tablet_manager.cc:1918] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:15:06.616605 1419 log.cc:1196] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/32013e6b6d4940348ffc86fc9cf26457
I20260806 02:15:06.616895 1419 ts_tablet_manager.cc:1939] T 32013e6b6d4940348ffc86fc9cf26457 P a3520d2a11934d058ae5dbdf278a476c: Deleting consensus metadata
I20260806 02:15:06.617276 1387 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260806 02:15:06.617393 1387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.000s
I20260806 02:15:06.617552 1387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:15:06.617581 1387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:06.626411 1387 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:44521
I20260806 02:15:06.626451 1540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:44521 every 8 connection(s)
I20260806 02:15:06.627329 1387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:15:06.633733 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 1387
I20260806 02:15:06.634141 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:46671
--local_ip_for_outbound_sockets=127.18.70.67
--tserver_master_addrs=127.18.70.126:38937
--webserver_port=39181
--webserver_interface=127.18.70.67
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 02:15:06 dist-test-slave-6k22 krb5kdc[559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982506, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:06.641413 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.629389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:38305 (local address 127.18.70.126:38937)
0806 02:15:06.629621 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:06.629625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:06.630474 (+ 849us) server_negotiation.cc:407] Connection header received
0806 02:15:06.631417 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:06.631420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:06.631461 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:06.631535 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:06.632678 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.633187 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.634425 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.634891 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.635007 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.637739 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:06.637742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:06.637744 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:06.637786 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:06.639203 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:06.639812 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:06.639815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:06.639817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:06.639903 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:06.640249 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:06.640252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:06.640254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:06.640445 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:06.640555 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:06.641084 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 02:15:06.641222 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260806 02:15:06.642205 1543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.629825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:38305)
0806 02:15:06.630320 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:06.630373 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:06.631224 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:06.631717 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:06.631723 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:06.632072 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:06.632549 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.632562 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.633302 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.633304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.634296 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.634307 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.635061 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.635064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.635202 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.635914 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:06.635942 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:06.637560 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:06.639331 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:06.639337 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:06.639351 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:06.639709 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:06.639998 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:06.640002 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:06.640004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:06.640147 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:06.640556 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:06.640563 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:06.640809 (+ 246us) client_negotiation.cc:769] Sending connection context
0806 02:15:06.641026 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 02:15:06.641350 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":141,"threads_started":1}
I20260806 02:15:06.643644 1541 heartbeater.cc:347] Connected to a master server at 127.18.70.126:38937
I20260806 02:15:06.643918 1541 heartbeater.cc:464] Registering TS with master...
I20260806 02:15:06.644423 1541 heartbeater.cc:511] Master 127.18.70.126:38937 requested a full tablet report, sending...
I20260806 02:15:06.645618 1178 ts_manager.cc:194] Registered new tserver with Master: a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521)
I20260806 02:15:06.646514 1178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:38305
W20260806 02:15:06.753612 1544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:06.753860 1544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:06.753902 1544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:06.757876 1544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:06.757925 1544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:06.757980 1544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:15:06.763324 1544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:46671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=39181
--tserver_master_addrs=127.18.70.126:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:06.764358 1544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:06.765349 1544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:06.775060 1549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:06.775079 1550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:06.775090 1552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:06.775178 1544 server_base.cc:1034] running on GCE node
I20260806 02:15:06.775822 1544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:06.776405 1544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:06.777575 1544 hybrid_clock.cc:648] HybridClock initialized: now 1785982506777559 us; error 25 us; skew 500 ppm
Aug 06 02:15:06 dist-test-slave-6k22 krb5kdc[559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982506, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:06.780272 1544 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:15:06.781091 1544 webserver.cc:460] Webserver started at http://127.18.70.67:39181/ using document root <none> and password file <none>
I20260806 02:15:06.781703 1544 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:06.781800 1544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:06.785580 1544 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 02:15:06.787705 1560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:06.788892 1544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:15:06.789026 1544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "1cb9f087a31040a68e69db6a93fc0398"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "7e7e4a230a59b1ac800825f241259704"
server_key_iv: "59a30b1631e2512fb9d9c3cc8c99e1e2"
server_key_version: "encryptionkey@0"
I20260806 02:15:06.789584 1544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:06.805408 1544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:06.808187 1544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:06.808389 1544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:06.808966 1544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:06.809814 1544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:15:06.809885 1544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:06.809958 1544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:15:06.810001 1544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:06.819234 1544 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:46671
I20260806 02:15:06.819255 1684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:46671 every 8 connection(s)
I20260806 02:15:06.820279 1544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:15:06.822235 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 1544
Aug 06 02:15:06 dist-test-slave-6k22 krb5kdc[559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982506, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:06.832062 1688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.822885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46166 (local address 127.18.70.126:38937)
0806 02:15:06.823110 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:06.823114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:06.823352 (+ 238us) server_negotiation.cc:407] Connection header received
0806 02:15:06.823529 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:06.823531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:06.823617 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:06.823712 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:06.824804 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.825486 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.826657 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.826970 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.827067 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.828603 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:06.828606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:06.828608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:06.828629 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:06.830228 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:06.830783 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:06.830786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:06.830787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:06.830870 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:06.831136 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:06.831139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:06.831141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:06.831311 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:06.831530 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:06.831704 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 02:15:06.831828 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":79,"threads_started":1}
I20260806 02:15:06.835196 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.822154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:51229 (local address 127.18.70.126:38937)
0806 02:15:06.822334 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:06.822340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:06.823244 (+ 904us) server_negotiation.cc:407] Connection header received
0806 02:15:06.825186 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:06.825189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:06.825234 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:06.825329 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:06.826677 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.827282 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.828935 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.829164 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.829272 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.831609 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:06.831617 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:06.831619 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:06.831640 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:06.833124 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:06.833668 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:06.833673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:06.833675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:06.833761 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:06.834107 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:06.834111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:06.834113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:06.834322 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:06.834446 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:06.834943 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 02:15:06.835050 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 02:15:06.836066 1687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.822544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:51229)
0806 02:15:06.823082 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:06.823145 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:06.824967 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:06.825457 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:06.825464 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:06.825956 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:06.826554 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.826573 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.827428 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.827431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.828804 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.828814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.829315 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.829319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.829490 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.829951 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:06.829967 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:06.831430 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:06.833268 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:06.833273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:06.833287 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:06.833560 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:06.833861 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:06.833864 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:06.833866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:06.834006 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:06.834447 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:06.834455 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:06.834689 (+ 234us) client_negotiation.cc:769] Sending connection context
0806 02:15:06.834897 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 02:15:06.835207 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":125,"threads_started":1}
I20260806 02:15:06.837141 1685 heartbeater.cc:347] Connected to a master server at 127.18.70.126:38937
I20260806 02:15:06.837419 1685 heartbeater.cc:464] Registering TS with master...
I20260806 02:15:06.837898 1685 heartbeater.cc:511] Master 127.18.70.126:38937 requested a full tablet report, sending...
I20260806 02:15:06.838891 1178 ts_manager.cc:194] Registered new tserver with Master: 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671)
I20260806 02:15:06.839433 1178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:51229
I20260806 02:15:06.845470 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:15:06.849882 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.846096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46168 (local address 127.18.70.126:38937)
0806 02:15:06.846234 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:06.846240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:06.846437 (+ 197us) server_negotiation.cc:407] Connection header received
0806 02:15:06.846634 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:06.846637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:06.846734 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:06.846849 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:15:06.847085 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.847628 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.848512 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.848737 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.848853 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.849163 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:15:06.849493 (+ 330us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:15:06.849621 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:06.849693 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 02:15:06.849741 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 02:15:06.850478 1178 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:46168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:15:06.853821 1178 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:15:06.861467 1308 tablet_service.cc:1467] Processing CreateTablet for tablet a8dc7ea321d543ac8278a0a5103e7926 (DEFAULT_TABLE table=test-table [id=778045405afe4454ac1dbdf77da90389]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:15:06.862206 1308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8dc7ea321d543ac8278a0a5103e7926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:06.865361 1264 tablet_bootstrap.cc:492] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115: Bootstrap starting.
I20260806 02:15:06.866142 1390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.860521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:46671 (local address 127.0.0.1:41132)
0806 02:15:06.860640 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:06.860656 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:06.860731 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:06.861737 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:06.861739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:06.861756 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:06.861866 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.861873 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.863402 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.863405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.864641 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.864652 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.865701 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.865705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.865799 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.865844 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 02:15:06.865934 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 02:15:06.866020 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 02:15:06.866184 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.860724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41132 (local address 127.18.70.67:46671)
0806 02:15:06.861190 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:06.861201 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:06.861225 (+ 24us) server_negotiation.cc:407] Connection header received
0806 02:15:06.861297 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:06.861302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:06.861560 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:06.862023 (+ 463us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:06.862171 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.863258 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.864776 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.865589 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.865701 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.865863 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:06.865965 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 02:15:06.866051 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":114,"threads_started":1}
I20260806 02:15:06.866178 1264 tablet_bootstrap.cc:654] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:06.867600 1264 tablet_bootstrap.cc:492] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115: No bootstrap required, opened a new log
I20260806 02:15:06.867691 1264 ts_tablet_manager.cc:1397] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 02:15:06.868158 1264 raft_consensus.cc:348] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:06.868281 1264 raft_consensus.cc:374] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:06.868321 1264 raft_consensus.cc:729] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f546d432630f400aa68ffd698cc36115, State: Initialized, Role: FOLLOWER
I20260806 02:15:06.868463 1264 consensus_queue.cc:260] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:06.868726 1264 ts_tablet_manager.cc:1428] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 02:15:06.869936 1615 tablet_service.cc:1467] Processing CreateTablet for tablet a8dc7ea321d543ac8278a0a5103e7926 (DEFAULT_TABLE table=test-table [id=778045405afe4454ac1dbdf77da90389]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:15:06.871037 1615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8dc7ea321d543ac8278a0a5103e7926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:06.877312 1700 tablet_bootstrap.cc:492] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398: Bootstrap starting.
I20260806 02:15:06.878911 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.861238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57394 (local address 127.18.70.66:44521)
0806 02:15:06.861540 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:06.861553 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:06.862189 (+ 636us) server_negotiation.cc:407] Connection header received
0806 02:15:06.862390 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:06.862394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:06.862569 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:06.862749 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:06.862942 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.864054 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.865230 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.878213 (+ 12983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.878369 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.878537 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:06.878598 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 02:15:06.878702 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":108,"threads_started":1}
I20260806 02:15:06.878927 1697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.860962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:44521 (local address 127.0.0.1:57394)
0806 02:15:06.862084 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:06.862099 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:06.862199 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:06.862724 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:06.862727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:06.862744 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:06.862845 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.862852 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.864207 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.864210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.865111 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.865120 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.878370 (+ 13250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.878374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.878472 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.878496 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 02:15:06.878568 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 02:15:06.878784 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"thread_start_us":89,"threads_started":1}
I20260806 02:15:06.879529 1700 tablet_bootstrap.cc:654] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:06.880512 1465 tablet_service.cc:1467] Processing CreateTablet for tablet a8dc7ea321d543ac8278a0a5103e7926 (DEFAULT_TABLE table=test-table [id=778045405afe4454ac1dbdf77da90389]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:15:06.880981 1700 log.cc:824] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:06.881526 1465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8dc7ea321d543ac8278a0a5103e7926. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:06.883103 1700 tablet_bootstrap.cc:492] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398: No bootstrap required, opened a new log
I20260806 02:15:06.883316 1700 ts_tablet_manager.cc:1397] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 02:15:06.886873 1419 tablet_bootstrap.cc:492] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c: Bootstrap starting.
I20260806 02:15:06.887311 1700 raft_consensus.cc:348] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:06.887617 1700 raft_consensus.cc:374] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:06.887681 1700 raft_consensus.cc:729] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb9f087a31040a68e69db6a93fc0398, State: Initialized, Role: FOLLOWER
I20260806 02:15:06.888111 1700 consensus_queue.cc:260] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:06.888775 1685 heartbeater.cc:503] Master 127.18.70.126:38937 was elected leader, sending a full tablet report...
I20260806 02:15:06.889047 1700 ts_tablet_manager.cc:1428] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260806 02:15:06.889070 1419 tablet_bootstrap.cc:654] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:06.889853 1419 log.cc:824] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:06.891891 1419 tablet_bootstrap.cc:492] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c: No bootstrap required, opened a new log
I20260806 02:15:06.892030 1419 ts_tablet_manager.cc:1397] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260806 02:15:06.894953 1419 raft_consensus.cc:348] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:06.895139 1419 raft_consensus.cc:374] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:06.895170 1419 raft_consensus.cc:729] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3520d2a11934d058ae5dbdf278a476c, State: Initialized, Role: FOLLOWER
I20260806 02:15:06.895521 1419 consensus_queue.cc:260] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:06.896173 1541 heartbeater.cc:503] Master 127.18.70.126:38937 was elected leader, sending a full tablet report...
I20260806 02:15:06.896371 1419 ts_tablet_manager.cc:1428] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c: Time spent starting tablet: real 0.004s user 0.002s sys 0.003s
W20260806 02:15:06.897723 1384 tablet.cc:2369] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:15:06.936002 1389 raft_consensus.cc:482] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:15:06.936228 1389 raft_consensus.cc:504] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:06.937438 1389 leader_election.cc:288] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671), a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521)
I20260806 02:15:06.941949 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.938071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:41751 (local address 127.18.70.66:44521)
0806 02:15:06.938213 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:06.938217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:06.938441 (+ 224us) server_negotiation.cc:407] Connection header received
0806 02:15:06.938680 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:06.938682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:06.938727 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:06.938766 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:06.939160 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.939725 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.940886 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.941318 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.941451 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.941506 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:06.941732 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 02:15:06.941803 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 02:15:06.941959 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.937788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:56147 (local address 127.18.70.67:46671)
0806 02:15:06.937946 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:06.937952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:06.938429 (+ 477us) server_negotiation.cc:407] Connection header received
0806 02:15:06.938648 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:06.938652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:06.938732 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:06.938829 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:06.939150 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.939685 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.940892 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.941342 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.941488 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.941549 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:06.941750 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 02:15:06.941824 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 02:15:06.941982 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.937793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:44521 (local address 127.18.70.65:41751)
0806 02:15:06.938313 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:06.938331 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:06.938521 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:06.938892 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:06.938894 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:06.938929 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:06.939046 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.939051 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.939839 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.939845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.940743 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.940753 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.941447 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.941451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.941554 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.941621 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 02:15:06.941714 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 02:15:06.941789 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":81,"spinlock_wait_cycles":2304,"thread_start_us":72,"threads_started":1}
I20260806 02:15:06.941987 1705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.937613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:46671 (local address 127.18.70.65:56147)
0806 02:15:06.938284 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:06.938298 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:06.938486 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:06.938845 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:06.938848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:06.938867 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:06.939028 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.939034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.939817 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.939824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.940757 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:06.940767 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.941478 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.941482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:06.941588 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.941640 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 02:15:06.941734 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 02:15:06.941812 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":26,"spinlock_wait_cycles":19200,"thread_start_us":101,"threads_started":1}
I20260806 02:15:06.942765 1489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8dc7ea321d543ac8278a0a5103e7926" candidate_uuid: "f546d432630f400aa68ffd698cc36115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3520d2a11934d058ae5dbdf278a476c" is_pre_election: true
I20260806 02:15:06.942859 1637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8dc7ea321d543ac8278a0a5103e7926" candidate_uuid: "f546d432630f400aa68ffd698cc36115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb9f087a31040a68e69db6a93fc0398" is_pre_election: true
I20260806 02:15:06.943105 1637 raft_consensus.cc:2454] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f546d432630f400aa68ffd698cc36115 in term 0.
I20260806 02:15:06.943105 1489 raft_consensus.cc:2454] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f546d432630f400aa68ffd698cc36115 in term 0.
I20260806 02:15:06.943650 1259 leader_election.cc:302] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3520d2a11934d058ae5dbdf278a476c, f546d432630f400aa68ffd698cc36115; no voters:
I20260806 02:15:06.943974 1389 raft_consensus.cc:2781] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:15:06.944072 1389 raft_consensus.cc:482] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:15:06.944134 1389 raft_consensus.cc:3037] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:06.945339 1389 raft_consensus.cc:504] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:06.945739 1389 leader_election.cc:288] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 1 election: Requested vote from peers 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671), a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521)
I20260806 02:15:06.946118 1637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8dc7ea321d543ac8278a0a5103e7926" candidate_uuid: "f546d432630f400aa68ffd698cc36115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb9f087a31040a68e69db6a93fc0398"
I20260806 02:15:06.946102 1489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8dc7ea321d543ac8278a0a5103e7926" candidate_uuid: "f546d432630f400aa68ffd698cc36115" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3520d2a11934d058ae5dbdf278a476c"
I20260806 02:15:06.946241 1637 raft_consensus.cc:3037] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:06.946241 1489 raft_consensus.cc:3037] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:06.947510 1637 raft_consensus.cc:2454] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f546d432630f400aa68ffd698cc36115 in term 1.
I20260806 02:15:06.947510 1489 raft_consensus.cc:2454] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f546d432630f400aa68ffd698cc36115 in term 1.
I20260806 02:15:06.947818 1259 leader_election.cc:302] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3520d2a11934d058ae5dbdf278a476c, f546d432630f400aa68ffd698cc36115; no voters:
I20260806 02:15:06.947966 1389 raft_consensus.cc:2781] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:15:06.948146 1389 raft_consensus.cc:686] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 LEADER]: Becoming Leader. State: Replica: f546d432630f400aa68ffd698cc36115, State: Running, Role: LEADER
I20260806 02:15:06.948423 1389 consensus_queue.cc:237] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:06.951380 1177 catalog_manager.cc:5495] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 reported cstate change: term changed from 0 to 1, leader changed from <none> to f546d432630f400aa68ffd698cc36115 (127.18.70.65). New cstate: current_term: 1 leader_uuid: "f546d432630f400aa68ffd698cc36115" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } health_report { overall_health: HEALTHY } } }
I20260806 02:15:06.966261 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:06.962600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48512 (local address 127.18.70.65:35841)
0806 02:15:06.962888 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:06.962896 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:06.962907 (+ 11us) server_negotiation.cc:407] Connection header received
0806 02:15:06.962968 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:06.962970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:06.963018 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:06.963148 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:15:06.963406 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.963985 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.964841 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:06.965062 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:06.965170 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:06.965451 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:15:06.965857 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:15:06.965986 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:06.966056 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 02:15:06.966109 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":87,"threads_started":1}
I20260806 02:15:06.970132 1489 raft_consensus.cc:1261] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c [term 1 FOLLOWER]: Refusing update from remote peer f546d432630f400aa68ffd698cc36115: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:15:06.970155 1637 raft_consensus.cc:1261] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398 [term 1 FOLLOWER]: Refusing update from remote peer f546d432630f400aa68ffd698cc36115: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:15:06.970871 1707 consensus_queue.cc:1035] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:15:06.971086 1389 consensus_queue.cc:1035] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:15:06.982036 1713 mvcc.cc:205] Tried to move back new op lower bound from 7315384348543688704 to 7315384348463534080. Current Snapshot: MvccSnapshot[applied={T|T < 7315384348543688704}]
I20260806 02:15:06.986229 1178 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:46168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:15:06.986400 1178 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:46168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:15:06.988997 1178 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Sending DeleteTablet for 3 replicas of tablet a8dc7ea321d543ac8278a0a5103e7926
I20260806 02:15:06.989686 1615 tablet_service.cc:1514] Processing DeleteTablet for tablet a8dc7ea321d543ac8278a0a5103e7926 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:15:06 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:41132
I20260806 02:15:06.989687 1465 tablet_service.cc:1514] Processing DeleteTablet for tablet a8dc7ea321d543ac8278a0a5103e7926 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:15:06 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:57394
I20260806 02:15:06.989902 1308 tablet_service.cc:1514] Processing DeleteTablet for tablet a8dc7ea321d543ac8278a0a5103e7926 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:15:06 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:48498
I20260806 02:15:06.990197 1723 tablet_replica.cc:330] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c: stopping tablet replica
I20260806 02:15:06.990258 1724 tablet_replica.cc:330] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115: stopping tablet replica
I20260806 02:15:06.990362 1724 raft_consensus.cc:2229] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:15:06.990398 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 1221
I20260806 02:15:06.991916 1723 raft_consensus.cc:2229] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:15:06.992165 1723 raft_consensus.cc:2258] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:15:06.993654 1723 ts_tablet_manager.cc:1905] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:15:06.993793 1722 tablet_replica.cc:330] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398: stopping tablet replica
I20260806 02:15:06.994068 1722 raft_consensus.cc:2229] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:15:06.994345 1722 raft_consensus.cc:2258] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:15:06.995795 1722 ts_tablet_manager.cc:1905] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:15:06.996217 1723 ts_tablet_manager.cc:1918] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:15:06.996313 1723 log.cc:1196] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/a8dc7ea321d543ac8278a0a5103e7926
I20260806 02:15:06.996578 1723 ts_tablet_manager.cc:1939] T a8dc7ea321d543ac8278a0a5103e7926 P a3520d2a11934d058ae5dbdf278a476c: Deleting consensus metadata
I20260806 02:15:06.997608 1163 catalog_manager.cc:4841] TS a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521): tablet a8dc7ea321d543ac8278a0a5103e7926 (table test-table [id=778045405afe4454ac1dbdf77da90389]) successfully deleted
I20260806 02:15:06.998252 1722 ts_tablet_manager.cc:1918] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:15:06.998380 1722 log.cc:1196] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/a8dc7ea321d543ac8278a0a5103e7926
I20260806 02:15:06.998744 1722 ts_tablet_manager.cc:1939] T a8dc7ea321d543ac8278a0a5103e7926 P 1cb9f087a31040a68e69db6a93fc0398: Deleting consensus metadata
I20260806 02:15:06.999809 1164 catalog_manager.cc:4841] TS 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671): tablet a8dc7ea321d543ac8278a0a5103e7926 (table test-table [id=778045405afe4454ac1dbdf77da90389]) successfully deleted
W20260806 02:15:07.000932 1165 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:35841 (error 108)
I20260806 02:15:07.001751 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 1387
I20260806 02:15:07.001967 1697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:07.001620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35841 (local address 127.0.0.1:48514)
0806 02:15:07.001740 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:07.001829 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:35841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 02:15:07.002753 1165 catalog_manager.cc:4584] TS f546d432630f400aa68ffd698cc36115 (127.18.70.65:35841): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a8dc7ea321d543ac8278a0a5103e7926: Network error: Client connection negotiation failed: client connection to 127.18.70.65:35841: connect: Connection refused (error 111)
I20260806 02:15:07.009032 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 1544
I20260806 02:15:07.015869 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 1145
I20260806 02:15:07.023777 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:38937
--webserver_interface=127.18.70.126
--webserver_port=32859
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:38937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:07.136022 1726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:07.136261 1726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:07.136298 1726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:07.140265 1726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:15:07.140316 1726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:07.140336 1726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:07.140353 1726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:15:07.140406 1726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:15:07.145555 1726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:38937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:38937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=32859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:07.146579 1726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:07.147658 1726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:07.156111 1734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:07.156111 1731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:07.156116 1732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:07.156396 1726 server_base.cc:1034] running on GCE node
I20260806 02:15:07.157017 1726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:07.158077 1726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:07.159267 1726 hybrid_clock.cc:648] HybridClock initialized: now 1785982507159246 us; error 34 us; skew 500 ppm
Aug 06 02:15:07 dist-test-slave-6k22 krb5kdc[559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982507, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:07.162182 1726 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:15:07.163174 1726 webserver.cc:460] Webserver started at http://127.18.70.126:32859/ using document root <none> and password file <none>
I20260806 02:15:07.163851 1726 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:07.163945 1726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:07.167744 1726 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260806 02:15:07.169849 1741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:07.171067 1726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:15:07.171208 1726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "6df0ca8b12504f7caee283811b0f92a7"
format_stamp: "Formatted at 2026-08-06 02:15:04 on dist-test-slave-6k22"
server_key: "cde4d3c984dd4bbf9c6c012d78fb277e"
server_key_iv: "a02144416e98256a0d0773df3cc992ad"
server_key_version: "encryptionkey@0"
I20260806 02:15:07.171816 1726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:07.192359 1726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:07.195624 1726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:07.195829 1726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:07.203137 1726 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:38937
I20260806 02:15:07.203404 1795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:38937 every 8 connection(s)
I20260806 02:15:07.204341 1726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:15:07.208520 1796 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:15:07.210084 1796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Bootstrap starting.
I20260806 02:15:07.210693 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 1726
I20260806 02:15:07.211109 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:35841
--local_ip_for_outbound_sockets=127.18.70.65
--tserver_master_addrs=127.18.70.126:38937
--webserver_port=44611
--webserver_interface=127.18.70.65
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:15:07.233458 1796 log.cc:824] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:07.246260 1796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:15:07.246719 1796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: Bootstrap complete.
I20260806 02:15:07.250911 1796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } }
I20260806 02:15:07.251322 1796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df0ca8b12504f7caee283811b0f92a7, State: Initialized, Role: FOLLOWER
I20260806 02:15:07.251868 1796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } }
I20260806 02:15:07.252022 1796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:15:07.252107 1796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:15:07.252219 1796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 2 FOLLOWER]: Advancing to term 3
I20260806 02:15:07.254045 1796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } }
I20260806 02:15:07.254384 1796 leader_election.cc:302] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6df0ca8b12504f7caee283811b0f92a7; no voters:
I20260806 02:15:07.254702 1796 leader_election.cc:288] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 02:15:07.254863 1801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 02:15:07.255095 1801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [term 3 LEADER]: Becoming Leader. State: Replica: 6df0ca8b12504f7caee283811b0f92a7, State: Running, Role: LEADER
I20260806 02:15:07.255388 1801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } }
I20260806 02:15:07.255970 1796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:15:07.256855 1803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6df0ca8b12504f7caee283811b0f92a7. Latest consensus state: current_term: 3 leader_uuid: "6df0ca8b12504f7caee283811b0f92a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } } }
I20260806 02:15:07.256995 1803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:07.257320 1802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6df0ca8b12504f7caee283811b0f92a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df0ca8b12504f7caee283811b0f92a7" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 38937 } } }
I20260806 02:15:07.257426 1802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7 [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:07.261406 1817 catalog_manager.cc:1259] Loaded cluster ID: 08212c9bf11e40678d3ba9c48aeceee7
I20260806 02:15:07.261493 1817 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: loading cluster ID for follower catalog manager: success
W20260806 02:15:07.262578 1817 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 6df0ca8b12504f7caee283811b0f92a7: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260806 02:15:07.262655 1817 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260806 02:15:07.262733 1810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:15:07.265703 1810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:15:07.266382 1810 catalog_manager.cc:1259] Loaded cluster ID: 08212c9bf11e40678d3ba9c48aeceee7
I20260806 02:15:07.266453 1810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 02:15:07.267390 1810 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f871d6a19ad google::LogMessage::Fail() at ??:0
@ 0x7f871d6a488a google::LogMessage::SendToLog() at ??:0
@ 0x7f871d6a14de google::LogMessage::Flush() at ??:0
@ 0x7f871d6a2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8727fe0fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f8727fe1d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8727fde579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f8728022640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f871f8f33d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f871f8f3c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f871f8f53f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f871f8e4512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8720f286db start_thread at ??:0
@ 0x7f871c19f61f clone at ??:0
W20260806 02:15:07.353315 1799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:07.353606 1799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:07.353725 1799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:07.357651 1799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:07.357725 1799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:07.357826 1799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:15:07.362879 1799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:35841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=44611
--tserver_master_addrs=127.18.70.126:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:07.364044 1799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:07.365091 1799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:07.374962 1843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:07.375041 1842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:07.375305 1799 server_base.cc:1034] running on GCE node
W20260806 02:15:07.375373 1845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:07.375844 1799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:07.376569 1799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:07.377784 1799 hybrid_clock.cc:648] HybridClock initialized: now 1785982507377752 us; error 31 us; skew 500 ppm
Aug 06 02:15:07 dist-test-slave-6k22 krb5kdc[559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982507, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:07.380702 1799 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:15:07.381619 1799 webserver.cc:460] Webserver started at http://127.18.70.65:44611/ using document root <none> and password file <none>
I20260806 02:15:07.382197 1799 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:07.382259 1799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:07.386310 1799 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 02:15:07.388677 1853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:07.390018 1799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:15:07.390152 1799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "f546d432630f400aa68ffd698cc36115"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "359ccfa6879a21be90bbac2be062d1ec"
server_key_iv: "79888f790f2428f97d1e7d373919cbe0"
server_key_version: "encryptionkey@0"
I20260806 02:15:07.390718 1799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:07.424584 1799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:07.428144 1799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:07.428390 1799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:07.429132 1799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:07.430418 1860 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 02:15:07.436090 1799 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 02:15:07.436165 1799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260806 02:15:07.436250 1799 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 02:15:07.437922 1799 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 02:15:07.437991 1799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 02:15:07.438349 1860 tablet_bootstrap.cc:492] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115: Bootstrap starting.
I20260806 02:15:07.449126 1799 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:35841
I20260806 02:15:07.449205 1972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:35841 every 8 connection(s)
I20260806 02:15:07.450183 1799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:15:07.459463 1860 log.cc:824] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:07.460268 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 1799
I20260806 02:15:07.460860 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:44521
--local_ip_for_outbound_sockets=127.18.70.66
--tserver_master_addrs=127.18.70.126:38937
--webserver_port=35783
--webserver_interface=127.18.70.66
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:15:07.471318 1860 tablet_bootstrap.cc:492] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:15:07.471789 1860 tablet_bootstrap.cc:492] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115: Bootstrap complete.
I20260806 02:15:07.473510 1860 ts_tablet_manager.cc:1397] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115: Time spent bootstrapping tablet: real 0.035s user 0.017s sys 0.013s
I20260806 02:15:07.477769 1860 raft_consensus.cc:348] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:07.478446 1860 raft_consensus.cc:729] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f546d432630f400aa68ffd698cc36115, State: Initialized, Role: FOLLOWER
I20260806 02:15:07.479058 1860 consensus_queue.cc:260] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:07.480556 1860 ts_tablet_manager.cc:1428] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
W20260806 02:15:07.487287 1976 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:15:07.452455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:45329)
0806 02:15:07.453063 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:07.453122 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:07.454226 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:07.486311 (+ 32085us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":161,"threads_started":1}
W20260806 02:15:07.487876 1973 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:38937 (0 consecutive failures): Network error: Failed to ping master at 127.18.70.126:38937: Client connection negotiation failed: client connection to 127.18.70.126:38937: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 02:15:07.488637 1976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:07.488347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:47137)
0806 02:15:07.488478 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:07.488554 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 02:15:07.489356 1976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:07.489134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:36063)
0806 02:15:07.489249 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:07.489299 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 02:15:07.489591 1973 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 02:15:07.585033 1978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:07.585304 1978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:07.585394 1978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:07.589452 1978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:07.589507 1978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:07.589562 1978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:15:07.594653 1978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:44521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=35783
--tserver_master_addrs=127.18.70.126:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:07.595772 1978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:07.596784 1978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:07.606590 1984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:07.606604 1985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:07.606640 1978 server_base.cc:1034] running on GCE node
W20260806 02:15:07.606647 1987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:07.607164 1978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:07.607878 1978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:07.609066 1978 hybrid_clock.cc:648] HybridClock initialized: now 1785982507609033 us; error 46 us; skew 500 ppm
Aug 06 02:15:07 dist-test-slave-6k22 krb5kdc[559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982507, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:07.611797 1978 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:15:07.612677 1978 webserver.cc:460] Webserver started at http://127.18.70.66:35783/ using document root <none> and password file <none>
I20260806 02:15:07.613246 1978 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:07.613343 1978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:07.617192 1978 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 02:15:07.619220 1995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:07.620484 1978 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:15:07.620622 1978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "a3520d2a11934d058ae5dbdf278a476c"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "bb71eccbca3bf9ab0dbbd70deb752ecf"
server_key_iv: "c322b5c92ecaefce2410b935d397787f"
server_key_version: "encryptionkey@0"
I20260806 02:15:07.621157 1978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:07.648144 1978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:07.650955 1978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:07.651194 1978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:07.651825 1978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:07.652659 1978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:15:07.652730 1978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:07.652801 1978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:15:07.652841 1978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:07.661970 1978 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:44521
I20260806 02:15:07.662000 2119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:44521 every 8 connection(s)
I20260806 02:15:07.663039 1978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:15:07.663617 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 1978
I20260806 02:15:07.664093 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:46671
--local_ip_for_outbound_sockets=127.18.70.67
--tserver_master_addrs=127.18.70.126:38937
--webserver_port=39181
--webserver_interface=127.18.70.67
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:15:07.668090 2124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:07.665608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:52767)
0806 02:15:07.666218 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:07.666375 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":160,"threads_started":1}
W20260806 02:15:07.668587 2120 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:38937 (0 consecutive failures): Network error: Failed to ping master at 127.18.70.126:38937: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
I20260806 02:15:07.669229 2124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:07.668990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:43653)
0806 02:15:07.669103 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:07.669175 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 02:15:07.669935 2124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:07.669657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:52169)
0806 02:15:07.669789 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:07.669853 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 02:15:07.670143 2120 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 02:15:07.780479 2123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:07.780707 2123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:07.780745 2123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:07.784555 2123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:07.784603 2123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:07.784663 2123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:15:07.789623 2123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:46671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=39181
--tserver_master_addrs=127.18.70.126:38937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:07.790585 2123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:07.791574 2123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:07.800980 2129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:07.801023 2133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:07.801021 2131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:07.801100 2123 server_base.cc:1034] running on GCE node
I20260806 02:15:07.801662 2123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:07.802248 2123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:07.803402 2123 hybrid_clock.cc:648] HybridClock initialized: now 1785982507803374 us; error 40 us; skew 500 ppm
Aug 06 02:15:07 dist-test-slave-6k22 krb5kdc[559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982507, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:07.806118 2123 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:15:07.806959 2123 webserver.cc:460] Webserver started at http://127.18.70.67:39181/ using document root <none> and password file <none>
I20260806 02:15:07.807504 2123 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:07.807633 2123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:07.811237 2123 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:15:07.813288 2141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:07.814502 2123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 02:15:07.814620 2123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "1cb9f087a31040a68e69db6a93fc0398"
format_stamp: "Formatted at 2026-08-06 02:15:05 on dist-test-slave-6k22"
server_key: "7e7e4a230a59b1ac800825f241259704"
server_key_iv: "59a30b1631e2512fb9d9c3cc8c99e1e2"
server_key_version: "encryptionkey@0"
I20260806 02:15:07.815162 2123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:07.830765 2123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:07.833956 2123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:07.834146 2123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:07.834717 2123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:07.835532 2123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:15:07.835638 2123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:07.835700 2123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:15:07.835736 2123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:07.844514 2123 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:46671
I20260806 02:15:07.844551 2262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:46671 every 8 connection(s)
I20260806 02:15:07.845489 2123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:15:07.849354 2265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:07.847624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:42431)
0806 02:15:07.848102 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:07.848218 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":114,"threads_started":1}
W20260806 02:15:07.849815 2263 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:38937 (0 consecutive failures): Network error: Failed to ping master at 127.18.70.126:38937: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
I20260806 02:15:07.850499 2265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:07.850242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:38727)
0806 02:15:07.850374 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:07.850430 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 02:15:07.851112 2265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:07.850906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:42629)
0806 02:15:07.851010 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:07.851054 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 02:15:07.851271 2263 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 02:15:07.852020 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 2123
I20260806 02:15:08.490944 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:08.490433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:35803)
0806 02:15:08.490755 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:08.490828 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":92,"threads_started":1}
I20260806 02:15:08.671193 2268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:08.670776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:55187)
0806 02:15:08.671036 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:08.671090 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":88,"threads_started":1}
I20260806 02:15:08.852190 2269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:08.851817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:51005)
0806 02:15:08.852038 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:08.852089 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":67,"threads_started":1}
I20260806 02:15:08.937987 2270 raft_consensus.cc:482] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:15:08.938223 2270 raft_consensus.cc:504] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:08.939373 2270 leader_election.cc:288] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671), a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521)
Aug 06 02:15:08 dist-test-slave-6k22 krb5kdc[559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982507, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.66@KRBTEST.COM
Aug 06 02:15:08 dist-test-slave-6k22 krb5kdc[559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982507, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.67@KRBTEST.COM
I20260806 02:15:08.953637 2273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:08.939600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:46671 (local address 127.18.70.65:46541)
0806 02:15:08.940318 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:08.940332 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:08.940440 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:08.941675 (+ 1235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:08.941678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:08.942202 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:08.942684 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:08.942699 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:08.944078 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:08.944082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:08.945645 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:08.945656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:08.947229 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:08.947233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:08.947326 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:08.947772 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:08.947792 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:08.949208 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:08.951175 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:08.951180 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:08.951182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:08.951471 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:08.951846 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:08.951865 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:08.951867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:08.952023 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:08.952633 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:08.952640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:08.952881 (+ 241us) client_negotiation.cc:769] Sending connection context
0806 02:15:08.953069 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 02:15:08.953406 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":453,"spinlock_wait_cycles":6784,"thread_start_us":66,"threads_started":1}
I20260806 02:15:08.953655 2272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:08.939863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:46541 (local address 127.18.70.67:46671)
0806 02:15:08.940285 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:08.940308 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:08.940430 (+ 122us) server_negotiation.cc:407] Connection header received
0806 02:15:08.941138 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:08.941143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:08.941444 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:08.941726 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:08.942881 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:08.943928 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:08.946499 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:08.947112 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:08.947226 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:08.949398 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:08.949403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:08.949414 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:08.949440 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:08.951038 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:08.951615 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:08.951620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:08.951624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:08.951738 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:08.952121 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:08.952125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:08.952127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:08.952510 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:08.952655 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:08.953182 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 02:15:08.953451 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":120,"threads_started":1}
I20260806 02:15:08.953905 2271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:08.939805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:43235 (local address 127.18.70.66:44521)
0806 02:15:08.940131 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:08.940159 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:08.940198 (+ 39us) server_negotiation.cc:407] Connection header received
0806 02:15:08.940892 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:08.940898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:08.941245 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:08.941568 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:08.942895 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:08.943950 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:08.945769 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:08.946239 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:08.946368 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:08.949042 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:08.949047 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:08.949059 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:08.949086 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:08.950938 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:08.951604 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:08.951610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:08.951622 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:08.951737 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:08.952112 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:08.952116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:08.952118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:08.952510 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:08.952656 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:08.953183 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 02:15:08.953769 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":131,"threads_started":1}
I20260806 02:15:08.953693 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:08.939600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:44521 (local address 127.18.70.65:43235)
0806 02:15:08.939711 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:08.939726 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:08.939837 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:08.941608 (+ 1771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:08.941614 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:08.942187 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:08.942685 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:08.942699 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:08.944123 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:08.944126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:08.945645 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:08.945656 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:08.946359 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:08.946362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:08.946507 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:08.947117 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:08.947133 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:08.948815 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:08.951064 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:08.951070 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:08.951101 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:08.951467 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:08.951851 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:08.951854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:08.951856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:08.952012 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:08.952633 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:08.952643 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:08.952877 (+ 234us) client_negotiation.cc:769] Sending connection context
0806 02:15:08.953067 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 02:15:08.953504 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":403,"spinlock_wait_cycles":8704}
W20260806 02:15:08.956789 1856 leader_election.cc:341] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671): Not found: Tablet not found: a8dc7ea321d543ac8278a0a5103e7926
W20260806 02:15:08.957007 1855 leader_election.cc:341] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521): Not found: Tablet not found: a8dc7ea321d543ac8278a0a5103e7926
I20260806 02:15:08.957156 1855 leader_election.cc:302] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f546d432630f400aa68ffd698cc36115; no voters: 1cb9f087a31040a68e69db6a93fc0398, a3520d2a11934d058ae5dbdf278a476c
I20260806 02:15:08.957448 2270 raft_consensus.cc:2726] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:15:09.492247 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:09.491827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:51755)
0806 02:15:09.492088 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:09.492145 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":74,"threads_started":1}
I20260806 02:15:09.672307 2275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:09.671922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:47847)
0806 02:15:09.672171 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:09.672222 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":71,"threads_started":1}
I20260806 02:15:09.853286 2276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:09.852894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:48291)
0806 02:15:09.853150 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:09.853200 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":87,"threads_started":1}
I20260806 02:15:10.493419 2277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:10.493019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:39049)
0806 02:15:10.493260 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:10.493321 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":76,"threads_started":1}
I20260806 02:15:10.673416 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:10.672998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:34533)
0806 02:15:10.673224 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:10.673275 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":67,"threads_started":1}
I20260806 02:15:10.833982 2279 raft_consensus.cc:482] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:15:10.834121 2279 raft_consensus.cc:504] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:10.834647 2279 leader_election.cc:288] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671), a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521)
W20260806 02:15:10.835526 1855 leader_election.cc:341] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521): Not found: Tablet not found: a8dc7ea321d543ac8278a0a5103e7926
W20260806 02:15:10.835664 1856 leader_election.cc:341] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671): Not found: Tablet not found: a8dc7ea321d543ac8278a0a5103e7926
I20260806 02:15:10.835752 1856 leader_election.cc:302] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f546d432630f400aa68ffd698cc36115; no voters: 1cb9f087a31040a68e69db6a93fc0398, a3520d2a11934d058ae5dbdf278a476c
I20260806 02:15:10.835911 2279 raft_consensus.cc:2726] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:15:10.854374 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:10.853991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:35083)
0806 02:15:10.854233 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:10.854286 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":78,"threads_started":1}
I20260806 02:15:11.494522 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:11.494125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:51117)
0806 02:15:11.494371 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:11.494424 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":77,"threads_started":1}
I20260806 02:15:11.674695 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:11.674307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:34177)
0806 02:15:11.674551 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:11.674604 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260806 02:15:11.855453 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:11.855043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:47905)
0806 02:15:11.855310 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:11.855361 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":93,"threads_started":1}
I20260806 02:15:12.495946 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:12.495425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:45411)
0806 02:15:12.495793 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:12.495849 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":138,"threads_started":1}
I20260806 02:15:12.676096 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:12.675584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:38275)
0806 02:15:12.675900 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:12.675974 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":87,"threads_started":1}
I20260806 02:15:12.856619 2286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:12.856225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:37277)
0806 02:15:12.856479 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:12.856531 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":92,"threads_started":1}
I20260806 02:15:13.497013 2287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:13.496623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:51075)
0806 02:15:13.496861 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:13.496914 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260806 02:15:13.568810 2288 raft_consensus.cc:482] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:15:13.568953 2288 raft_consensus.cc:504] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:13.569337 2288 leader_election.cc:288] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671), a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521)
W20260806 02:15:13.570210 1855 leader_election.cc:341] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521): Not found: Tablet not found: a8dc7ea321d543ac8278a0a5103e7926
W20260806 02:15:13.570328 1856 leader_election.cc:341] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671): Not found: Tablet not found: a8dc7ea321d543ac8278a0a5103e7926
I20260806 02:15:13.570472 1856 leader_election.cc:302] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f546d432630f400aa68ffd698cc36115; no voters: 1cb9f087a31040a68e69db6a93fc0398, a3520d2a11934d058ae5dbdf278a476c
I20260806 02:15:13.570684 2288 raft_consensus.cc:2726] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:15:13.677253 2289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:13.676875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:50979)
0806 02:15:13.677112 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:13.677165 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":71,"threads_started":1}
I20260806 02:15:13.857764 2290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:13.857356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:34265)
0806 02:15:13.857616 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:13.857671 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":95,"threads_started":1}
I20260806 02:15:14.498214 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:14.497760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:37713)
0806 02:15:14.498041 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:14.498113 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":76,"threads_started":1}
I20260806 02:15:14.678438 2292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:14.677994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:54541)
0806 02:15:14.678277 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:14.678329 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":73,"threads_started":1}
I20260806 02:15:14.859208 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:14.858709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:45833)
0806 02:15:14.859051 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:14.859112 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":100,"threads_started":1}
I20260806 02:15:15.127395 2294 raft_consensus.cc:482] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:15:15.127527 2294 raft_consensus.cc:504] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:15.127978 2294 leader_election.cc:288] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671), a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521)
W20260806 02:15:15.128770 1855 leader_election.cc:341] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521): Not found: Tablet not found: a8dc7ea321d543ac8278a0a5103e7926
W20260806 02:15:15.128878 1856 leader_election.cc:341] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671): Not found: Tablet not found: a8dc7ea321d543ac8278a0a5103e7926
I20260806 02:15:15.128999 1856 leader_election.cc:302] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f546d432630f400aa68ffd698cc36115; no voters: 1cb9f087a31040a68e69db6a93fc0398, a3520d2a11934d058ae5dbdf278a476c
I20260806 02:15:15.129171 2294 raft_consensus.cc:2726] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:15:15.499416 2295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:15.498984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:54725)
0806 02:15:15.499256 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:15.499307 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":108,"threads_started":1}
I20260806 02:15:15.679771 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:15.679266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:57657)
0806 02:15:15.679602 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:15.679672 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":107,"threads_started":1}
I20260806 02:15:15.860394 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:15.859980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:53735)
0806 02:15:15.860226 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:15.860280 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":70,"threads_started":1}
I20260806 02:15:16.500613 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:16.500194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:46117)
0806 02:15:16.500463 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:16.500516 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":85,"threads_started":1}
I20260806 02:15:16.681025 2299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:16.680577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:51933)
0806 02:15:16.680870 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:16.680926 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":82,"threads_started":1}
I20260806 02:15:16.861466 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:16.861079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:54269)
0806 02:15:16.861331 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:16.861379 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":91,"threads_started":1}
I20260806 02:15:17.501837 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:17.501453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:35457)
0806 02:15:17.501687 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:17.501738 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":72,"threads_started":1}
I20260806 02:15:17.682271 2302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:17.681790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:42589)
0806 02:15:17.682119 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:17.682175 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":73,"threads_started":1}
I20260806 02:15:17.862692 2303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:17.862328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:43795)
0806 02:15:17.862554 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:17.862604 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":70,"threads_started":1}
I20260806 02:15:18.502951 2304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:18.502561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:52849)
0806 02:15:18.502797 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:18.502850 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":73,"threads_started":1}
I20260806 02:15:18.683586 2306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:18.683108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:34479)
0806 02:15:18.683417 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:18.683468 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":125,"threads_started":1}
I20260806 02:15:18.863849 2307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:18.863380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:51111)
0806 02:15:18.863700 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:18.863755 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":89,"threads_started":1}
I20260806 02:15:19.504158 2308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:19.503750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:37217)
0806 02:15:19.504003 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:19.504057 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":79,"threads_started":1}
I20260806 02:15:19.684733 2309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:19.684314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:38737)
0806 02:15:19.684583 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:19.684636 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":96,"threads_started":1}
I20260806 02:15:19.865057 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:19.864623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:60563)
0806 02:15:19.864892 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:19.864956 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":90,"threads_started":1}
I20260806 02:15:20.505508 2312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:20.504992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:35335)
0806 02:15:20.505305 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:20.505378 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":99,"threads_started":1}
I20260806 02:15:20.685880 2313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:20.685452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:38643)
0806 02:15:20.685717 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:20.685784 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":94,"threads_started":1}
I20260806 02:15:20.784173 2314 raft_consensus.cc:482] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:15:20.784307 2314 raft_consensus.cc:504] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cb9f087a31040a68e69db6a93fc0398" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 46671 } } peers { permanent_uuid: "a3520d2a11934d058ae5dbdf278a476c" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44521 } } peers { permanent_uuid: "f546d432630f400aa68ffd698cc36115" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35841 } }
I20260806 02:15:20.784687 2314 leader_election.cc:288] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671), a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521)
W20260806 02:15:20.785532 1856 leader_election.cc:341] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1cb9f087a31040a68e69db6a93fc0398 (127.18.70.67:46671): Not found: Tablet not found: a8dc7ea321d543ac8278a0a5103e7926
W20260806 02:15:20.785640 1855 leader_election.cc:341] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3520d2a11934d058ae5dbdf278a476c (127.18.70.66:44521): Not found: Tablet not found: a8dc7ea321d543ac8278a0a5103e7926
I20260806 02:15:20.785717 1855 leader_election.cc:302] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f546d432630f400aa68ffd698cc36115; no voters: 1cb9f087a31040a68e69db6a93fc0398, a3520d2a11934d058ae5dbdf278a476c
I20260806 02:15:20.785857 2314 raft_consensus.cc:2726] T a8dc7ea321d543ac8278a0a5103e7926 P f546d432630f400aa68ffd698cc36115 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:15:20.866313 2315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:20.865906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:44475)
0806 02:15:20.866165 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:20.866219 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":93,"threads_started":1}
I20260806 02:15:21.506686 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:21.506276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:56595)
0806 02:15:21.506529 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:21.506589 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":101,"threads_started":1}
I20260806 02:15:21.687037 2318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:21.686569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:32997)
0806 02:15:21.686868 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:21.686922 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":97,"threads_started":1}
I20260806 02:15:21.867470 2319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:21.867044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:40465)
0806 02:15:21.867304 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:21.867369 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":83,"threads_started":1}
I20260806 02:15:22.508195 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:22.507805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.65:51679)
0806 02:15:22.508045 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:22.508098 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":74,"threads_started":1}
I20260806 02:15:22.688297 2321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:22.687924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.66:38989)
0806 02:15:22.688155 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:22.688204 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":70,"threads_started":1}
I20260806 02:15:22.857620 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 1799
I20260806 02:15:22.868848 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:22.868442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:38937 (local address 127.18.70.67:42193)
0806 02:15:22.868684 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:22.868747 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":82,"threads_started":1}
I20260806 02:15:22.870445 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 1978
I20260806 02:15:22.877322 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 2123
2026-08-06T02:15:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19273 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:15:22 dist-test-slave-6k22 krb5kdc[2326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:15:22 dist-test-slave-6k22 krb5kdc[2326](info): set up 2 sockets
Aug 06 02:15:22 dist-test-slave-6k22 krb5kdc[2326](info): commencing operation
krb5kdc: starting...
W20260806 02:15:24.017417 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:15:24 dist-test-slave-6k22 krb5kdc[2326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:15:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:15:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:15:24.138520 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:35757
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:35757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:24.252584 2340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:24.252813 2340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:24.252851 2340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:24.256767 2340 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 02:15:24.256809 2340 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 02:15:24.256825 2340 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 02:15:24.256839 2340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:15:24.256856 2340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:24.256873 2340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:24.256888 2340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:15:24.256903 2340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:15:24.262091 2340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:35757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:35757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:24.263078 2340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:24.264081 2340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:24.272576 2348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:24.272570 2347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:24.272610 2350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:24.272938 2340 server_base.cc:1034] running on GCE node
I20260806 02:15:24.273480 2340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:24.274561 2340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:24.275743 2340 hybrid_clock.cc:648] HybridClock initialized: now 1785982524275711 us; error 42 us; skew 500 ppm
Aug 06 02:15:24 dist-test-slave-6k22 krb5kdc[2326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982524, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:24.278439 2340 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:15:24.279342 2340 webserver.cc:460] Webserver started at http://127.18.70.126:40031/ using document root <none> and password file <none>
I20260806 02:15:24.279974 2340 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:24.280066 2340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:24.280304 2340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:15:24.282325 2340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "6a71232f5f23d82bb6eae54d0e764ce2"
server_key_iv: "c07e18b46fac918e8b70c4924429ffd4"
server_key_version: "encryptionkey@0"
I20260806 02:15:24.282899 2340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "6a71232f5f23d82bb6eae54d0e764ce2"
server_key_iv: "c07e18b46fac918e8b70c4924429ffd4"
server_key_version: "encryptionkey@0"
I20260806 02:15:24.286592 2340 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 02:15:24.288973 2359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:24.290184 2340 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:15:24.290323 2340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "6a71232f5f23d82bb6eae54d0e764ce2"
server_key_iv: "c07e18b46fac918e8b70c4924429ffd4"
server_key_version: "encryptionkey@0"
I20260806 02:15:24.290441 2340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:24.311247 2340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:24.314067 2340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:24.314273 2340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:24.321621 2340 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:35757
I20260806 02:15:24.321918 2415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:35757 every 8 connection(s)
I20260806 02:15:24.322659 2340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:15:24.324321 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 2340
I20260806 02:15:24.324465 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:15:24.324805 18713 external_mini_cluster.cc:1428] Setting key 405b09057509f2019cc0cf67245c66c8
I20260806 02:15:24.325989 2416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:24.332698 2416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Bootstrap starting.
Aug 06 02:15:24 dist-test-slave-6k22 krb5kdc[2326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:24.335408 2416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:24.336181 2416 log.cc:824] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:24.338322 2416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: No bootstrap required, opened a new log
I20260806 02:15:24.339282 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:24.326197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52222 (local address 127.18.70.126:35757)
0806 02:15:24.326624 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:24.326640 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:24.326677 (+ 37us) server_negotiation.cc:407] Connection header received
0806 02:15:24.327323 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:24.327339 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:24.327740 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:24.328024 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:24.328954 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:24.329995 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:24.331160 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:24.331693 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:24.331810 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:24.334548 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:24.334553 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:24.334569 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:24.334600 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:24.336456 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:24.336917 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:24.336936 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:24.336943 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:24.337056 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:24.337348 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:24.337351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:24.337353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:24.337717 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:24.337868 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:24.338181 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 02:15:24.338574 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":143,"threads_started":1}
I20260806 02:15:24.342042 2416 raft_consensus.cc:348] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } }
I20260806 02:15:24.342296 2416 raft_consensus.cc:374] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:24.342389 2416 raft_consensus.cc:729] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e370ecdf144c4b5facc77b7ecbc9c2cc, State: Initialized, Role: FOLLOWER
I20260806 02:15:24.342803 2416 consensus_queue.cc:260] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } }
I20260806 02:15:24.342947 2416 raft_consensus.cc:388] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:15:24.343030 2416 raft_consensus.cc:482] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:15:24.343145 2416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:24.344058 2416 raft_consensus.cc:504] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } }
I20260806 02:15:24.344381 2416 leader_election.cc:302] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e370ecdf144c4b5facc77b7ecbc9c2cc; no voters:
I20260806 02:15:24.344749 2416 leader_election.cc:288] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:15:24.344831 2422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:15:24.345124 2422 raft_consensus.cc:686] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 1 LEADER]: Becoming Leader. State: Replica: e370ecdf144c4b5facc77b7ecbc9c2cc, State: Running, Role: LEADER
I20260806 02:15:24.345512 2422 consensus_queue.cc:237] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } }
I20260806 02:15:24.346052 2416 sys_catalog.cc:564] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:15:24.347270 2425 sys_catalog.cc:455] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader e370ecdf144c4b5facc77b7ecbc9c2cc. Latest consensus state: current_term: 1 leader_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } } }
I20260806 02:15:24.347397 2425 sys_catalog.cc:458] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:24.347841 2433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:15:24.347790 2423 sys_catalog.cc:455] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } } }
I20260806 02:15:24.347985 2423 sys_catalog.cc:458] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:24.350766 2433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:15:24.356875 2433 catalog_manager.cc:1347] Generated new cluster ID: c8298e54946c484580cd31c97823b030
I20260806 02:15:24.356935 2433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:15:24.377864 2433 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:15:24.379145 2433 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:15:24.393213 2433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Generated new TSK 0
I20260806 02:15:24.394006 2433 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:15:24.426857 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35757
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:24.553071 2446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:24.553316 2446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:24.553386 2446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:24.557281 2446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:24.557329 2446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:24.557435 2446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:15:24.562549 2446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:24.563606 2446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:24.564618 2446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:24.574349 2453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:24.574383 2455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:24.574527 2446 server_base.cc:1034] running on GCE node
W20260806 02:15:24.574357 2452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:24.575035 2446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:24.575752 2446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:24.576942 2446 hybrid_clock.cc:648] HybridClock initialized: now 1785982524576927 us; error 25 us; skew 500 ppm
Aug 06 02:15:24 dist-test-slave-6k22 krb5kdc[2326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982524, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:24.579677 2446 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:15:24.580504 2446 webserver.cc:460] Webserver started at http://127.18.70.65:35553/ using document root <none> and password file <none>
I20260806 02:15:24.581081 2446 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:24.581173 2446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:24.581410 2446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:15:24.583353 2446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "9f7277bf61e4409080ccc39f9540a80f"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "5213d070314541d6d147277d1e58ed77"
server_key_iv: "f18ce112d48718eab89c9c92f329ca99"
server_key_version: "encryptionkey@0"
I20260806 02:15:24.583985 2446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "9f7277bf61e4409080ccc39f9540a80f"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "5213d070314541d6d147277d1e58ed77"
server_key_iv: "f18ce112d48718eab89c9c92f329ca99"
server_key_version: "encryptionkey@0"
I20260806 02:15:24.587867 2446 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 02:15:24.590276 2462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:24.591637 2446 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:15:24.591761 2446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "9f7277bf61e4409080ccc39f9540a80f"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "5213d070314541d6d147277d1e58ed77"
server_key_iv: "f18ce112d48718eab89c9c92f329ca99"
server_key_version: "encryptionkey@0"
I20260806 02:15:24.591856 2446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:24.609761 2446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:24.612499 2446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:24.612648 2446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:24.613157 2446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:24.614014 2446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:15:24.614090 2446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:24.614161 2446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:15:24.614215 2446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:24.622823 2446 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:40809
I20260806 02:15:24.622846 2588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:40809 every 8 connection(s)
I20260806 02:15:24.623816 2446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
Aug 06 02:15:24 dist-test-slave-6k22 krb5kdc[2326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982524, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:24.634023 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 2446
I20260806 02:15:24.634167 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:15:24.634541 18713 external_mini_cluster.cc:1428] Setting key 7839fa5a1b6f6bfcfb6d0d573472c75d
I20260806 02:15:24.637697 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:24.625572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:42691 (local address 127.18.70.126:35757)
0806 02:15:24.625717 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:24.625721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:24.626555 (+ 834us) server_negotiation.cc:407] Connection header received
0806 02:15:24.627953 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:24.627957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:24.628019 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:24.628125 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:24.629301 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:24.629829 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:24.630978 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:24.631211 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:24.631319 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:24.633998 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:24.634005 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:24.634007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:24.634041 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:24.635544 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:24.636139 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:24.636143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:24.636144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:24.636240 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:24.636625 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:24.636627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:24.636628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:24.636791 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:24.636866 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:24.637475 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 02:15:24.637581 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 02:15:24.638814 2593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:24.625945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:42691)
0806 02:15:24.626396 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:24.626439 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:24.627723 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:24.628304 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:24.628311 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:24.628650 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:24.629142 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:24.629155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:24.629984 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:24.629989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:24.630843 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:24.630854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:24.631390 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:24.631394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:24.631589 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:24.632182 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:24.632198 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:24.633820 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:24.635689 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:24.635694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:24.635705 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:24.636040 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:24.636347 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:24.636350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:24.636352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:24.636535 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:24.636914 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:24.636923 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:24.637231 (+ 308us) client_negotiation.cc:769] Sending connection context
0806 02:15:24.637439 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 02:15:24.637777 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":131,"threads_started":1}
I20260806 02:15:24.640179 2590 heartbeater.cc:347] Connected to a master server at 127.18.70.126:35757
I20260806 02:15:24.640446 2590 heartbeater.cc:464] Registering TS with master...
I20260806 02:15:24.641098 2590 heartbeater.cc:511] Master 127.18.70.126:35757 requested a full tablet report, sending...
I20260806 02:15:24.642828 2377 ts_manager.cc:194] Registered new tserver with Master: 9f7277bf61e4409080ccc39f9540a80f (127.18.70.65:40809)
I20260806 02:15:24.644114 2377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:42691
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:15:24.664175 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35757
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:24.775254 2596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:24.775482 2596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:24.775523 2596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:24.779353 2596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:24.779403 2596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:24.779458 2596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:15:24.784510 2596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:24.785422 2596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:24.786448 2596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:24.795805 2604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:24.795816 2601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:24.795816 2602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:24.796032 2596 server_base.cc:1034] running on GCE node
I20260806 02:15:24.796610 2596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:24.797230 2596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:24.798432 2596 hybrid_clock.cc:648] HybridClock initialized: now 1785982524798410 us; error 33 us; skew 500 ppm
Aug 06 02:15:24 dist-test-slave-6k22 krb5kdc[2326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982524, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:24.801120 2596 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:15:24.801934 2596 webserver.cc:460] Webserver started at http://127.18.70.66:34571/ using document root <none> and password file <none>
I20260806 02:15:24.802482 2596 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:24.802590 2596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:24.802832 2596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:15:24.805002 2596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "6fdec9444a874f5185a7ae5a8d3a5611"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "e85c266fc46503137064412a508f4343"
server_key_iv: "27a9d48d6c579fd9323206893489e047"
server_key_version: "encryptionkey@0"
I20260806 02:15:24.805560 2596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "6fdec9444a874f5185a7ae5a8d3a5611"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "e85c266fc46503137064412a508f4343"
server_key_iv: "27a9d48d6c579fd9323206893489e047"
server_key_version: "encryptionkey@0"
I20260806 02:15:24.809145 2596 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:15:24.811538 2612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:24.812798 2596 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:15:24.812940 2596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "6fdec9444a874f5185a7ae5a8d3a5611"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "e85c266fc46503137064412a508f4343"
server_key_iv: "27a9d48d6c579fd9323206893489e047"
server_key_version: "encryptionkey@0"
I20260806 02:15:24.813054 2596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:24.823247 2596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:24.825929 2596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:24.826140 2596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:24.826696 2596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:24.827494 2596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:15:24.827564 2596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:24.827672 2596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:15:24.827724 2596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:24.836748 2596 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:42689
I20260806 02:15:24.836779 2735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:42689 every 8 connection(s)
I20260806 02:15:24.837604 2596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:15:24.840411 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 2596
I20260806 02:15:24.840536 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:15:24.840847 18713 external_mini_cluster.cc:1428] Setting key c2760c45ee4f29395a4e6b007aa56969
Aug 06 02:15:24 dist-test-slave-6k22 krb5kdc[2326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982524, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
I20260806 02:15:24.852528 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:24.839303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:42763 (local address 127.18.70.126:35757)
0806 02:15:24.839469 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:24.839474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:24.840914 (+ 1440us) server_negotiation.cc:407] Connection header received
0806 02:15:24.841973 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:24.841976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:24.842031 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:24.842135 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:24.843298 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:24.843913 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:24.845054 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:24.845276 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:24.845394 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:24.848228 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:24.848232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:24.848236 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:24.848266 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:24.850001 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:24.850515 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:24.850520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:24.850525 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:24.850625 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:24.851146 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:24.851149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:24.851151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:24.851415 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:24.851531 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:24.852142 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 02:15:24.852280 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 02:15:24.853310 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:24.839748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:42763)
0806 02:15:24.840761 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:24.840812 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:24.841753 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:24.842320 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:24.842327 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:24.842691 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:24.843149 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:24.843161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:24.844045 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:24.844048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:24.844923 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:24.844933 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:24.845434 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:24.845438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:24.845603 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:24.846161 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:24.846187 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:24.848041 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:24.850132 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:24.850136 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:24.850148 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:24.850409 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:24.850725 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:24.850727 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:24.850729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:24.851008 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:24.851552 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:24.851628 (+ 76us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:24.851871 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 02:15:24.852055 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 02:15:24.852330 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":667,"threads_started":1}
I20260806 02:15:24.854507 2736 heartbeater.cc:347] Connected to a master server at 127.18.70.126:35757
I20260806 02:15:24.854933 2736 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
I20260806 02:15:24.855496 2736 heartbeater.cc:511] Master 127.18.70.126:35757 requested a full tablet report, sending...
I20260806 02:15:24.856628 2377 ts_manager.cc:194] Registered new tserver with Master: 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689)
I20260806 02:15:24.857439 2377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:42763
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:15:24.872001 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35757
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:24.985329 2741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:24.985565 2741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:24.985606 2741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:24.989578 2741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:24.989626 2741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:24.989719 2741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:15:24.994781 2741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:24.995779 2741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:24.996790 2741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:25.006542 2747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:25.006543 2746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:25.006567 2749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:25.006595 2741 server_base.cc:1034] running on GCE node
I20260806 02:15:25.007171 2741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:25.007881 2741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:25.009038 2741 hybrid_clock.cc:648] HybridClock initialized: now 1785982525009005 us; error 45 us; skew 500 ppm
Aug 06 02:15:25 dist-test-slave-6k22 krb5kdc[2326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982525, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:25.011871 2741 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:15:25.012761 2741 webserver.cc:460] Webserver started at http://127.18.70.67:38177/ using document root <none> and password file <none>
I20260806 02:15:25.013295 2741 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:25.013357 2741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:25.013530 2741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:15:25.015513 2741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "d502f01a8fd44cd08a55090c1958e1cf"
format_stamp: "Formatted at 2026-08-06 02:15:25 on dist-test-slave-6k22"
server_key: "0483c2fd2ad70904616567362629dbf0"
server_key_iv: "e79609d1f2111060f6186231de428dac"
server_key_version: "encryptionkey@0"
I20260806 02:15:25.016144 2741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "d502f01a8fd44cd08a55090c1958e1cf"
format_stamp: "Formatted at 2026-08-06 02:15:25 on dist-test-slave-6k22"
server_key: "0483c2fd2ad70904616567362629dbf0"
server_key_iv: "e79609d1f2111060f6186231de428dac"
server_key_version: "encryptionkey@0"
I20260806 02:15:25.019796 2741 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 02:15:25.022145 2756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:25.023406 2741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:15:25.023541 2741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "d502f01a8fd44cd08a55090c1958e1cf"
format_stamp: "Formatted at 2026-08-06 02:15:25 on dist-test-slave-6k22"
server_key: "0483c2fd2ad70904616567362629dbf0"
server_key_iv: "e79609d1f2111060f6186231de428dac"
server_key_version: "encryptionkey@0"
I20260806 02:15:25.023700 2741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:25.043157 2741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:25.046425 2741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:25.046617 2741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:25.047314 2741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:25.048264 2741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:15:25.048310 2741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:25.048350 2741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:15:25.048367 2741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:25.057133 2741 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:35737
I20260806 02:15:25.057158 2878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:35737 every 8 connection(s)
I20260806 02:15:25.058001 2741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
Aug 06 02:15:25 dist-test-slave-6k22 krb5kdc[2326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982525, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:25.068325 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 2741
I20260806 02:15:25.068488 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:15:25.068903 18713 external_mini_cluster.cc:1428] Setting key 2ea9e8d700fd232e4b4f4d1c0c03f1da
I20260806 02:15:25.071745 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.059763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:43815 (local address 127.18.70.126:35757)
0806 02:15:25.059906 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:25.059909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:25.060738 (+ 829us) server_negotiation.cc:407] Connection header received
0806 02:15:25.061988 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:25.061991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:25.062038 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:25.062112 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:25.063225 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.063812 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.064892 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.065114 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.065193 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.067545 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:25.067548 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:25.067550 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:25.067617 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:25.069337 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:25.070042 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:25.070046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:25.070047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:25.070136 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:25.070511 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:25.070516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:25.070523 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:25.070718 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:25.070829 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:25.071424 (+ 595us) server_negotiation.cc:299] Negotiation successful
0806 02:15:25.071546 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:15:25.072463 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.060128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:43815)
0806 02:15:25.060572 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:25.060617 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:25.061784 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:25.062239 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:25.062245 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:25.062565 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:25.063035 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:25.063048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.063927 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.063930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:25.064784 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:25.064793 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.065241 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.065244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:25.065382 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.065943 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:25.065959 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:25.067381 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:25.069451 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:25.069460 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:25.069481 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:25.069726 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:25.070249 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:25.070252 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:25.070254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:25.070400 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:25.070829 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:25.070835 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:25.071148 (+ 313us) client_negotiation.cc:769] Sending connection context
0806 02:15:25.071354 (+ 206us) client_negotiation.cc:240] Negotiation successful
0806 02:15:25.071693 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":124,"threads_started":1}
I20260806 02:15:25.073482 2879 heartbeater.cc:347] Connected to a master server at 127.18.70.126:35757
I20260806 02:15:25.073720 2879 heartbeater.cc:464] Registering TS with master...
I20260806 02:15:25.074199 2879 heartbeater.cc:511] Master 127.18.70.126:35757 requested a full tablet report, sending...
I20260806 02:15:25.075176 2377 ts_manager.cc:194] Registered new tserver with Master: d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737)
I20260806 02:15:25.075793 2377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:43815
I20260806 02:15:25.082626 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:15:25.091851 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.084579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52232 (local address 127.18.70.126:35757)
0806 02:15:25.084711 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:25.084717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:25.084937 (+ 220us) server_negotiation.cc:407] Connection header received
0806 02:15:25.085098 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:25.085100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:25.085137 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:25.085216 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:25.085913 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.086412 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.087228 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.087437 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.087539 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.088939 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:25.088943 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:25.088946 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:25.088976 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:25.090113 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:25.090542 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:25.090545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:25.090547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:25.090627 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:25.090926 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:25.090929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:25.090931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:25.091139 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:25.091282 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:25.091520 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 02:15:25.091660 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 02:15:25.094615 2377 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:15:25.097801 2377 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:15:25.112259 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.106891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43446 (local address 127.18.70.67:35737)
0806 02:15:25.107327 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:25.107342 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:25.107811 (+ 469us) server_negotiation.cc:407] Connection header received
0806 02:15:25.108023 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:25.108027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:25.108167 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:25.108291 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:25.108707 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.109700 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.110977 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.111598 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.111722 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.111876 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:25.112039 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 02:15:25.112127 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":136,"threads_started":1}
I20260806 02:15:25.112250 2895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.106681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:35737 (local address 127.0.0.1:43446)
0806 02:15:25.107726 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:25.107738 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:25.107886 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:25.108467 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:25.108470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:25.108487 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:25.108590 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:25.108596 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.109841 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.109843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:25.110839 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:25.110859 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.111723 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.111727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:25.111836 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.111873 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 02:15:25.111996 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 02:15:25.112114 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":992,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260806 02:15:25.112573 2892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.106318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:40809 (local address 127.0.0.1:48264)
0806 02:15:25.107067 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:25.107114 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:25.107763 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:25.108268 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:25.108271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:25.108293 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:25.108428 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:25.108436 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.109390 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.109395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:25.110711 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:25.110721 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.112063 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.112067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:25.112223 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.112276 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 02:15:25.112343 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 02:15:25.112426 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":153,"spinlock_wait_cycles":1408,"thread_start_us":65,"threads_started":1}
I20260806 02:15:25.112686 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.107100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42436 (local address 127.18.70.66:42689)
0806 02:15:25.107524 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:25.107542 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:25.107593 (+ 51us) server_negotiation.cc:407] Connection header received
0806 02:15:25.107676 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:25.107681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:25.107895 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:25.108045 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:25.108574 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.109643 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.111055 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.111799 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.111897 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.112000 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:25.112424 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 02:15:25.112531 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":100,"threads_started":1}
I20260806 02:15:25.112989 2893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.106411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:42689 (local address 127.0.0.1:42436)
0806 02:15:25.107359 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:25.107375 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:25.107484 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:25.108198 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:25.108203 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:25.108246 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:25.108452 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:25.108459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.109798 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.109801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:25.110940 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:25.110950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.112063 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.112067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:25.112226 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.112275 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 02:15:25.112343 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 02:15:25.112870 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"mutex_wait_us":549,"thread_start_us":122,"threads_started":1}
I20260806 02:15:25.113085 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.106441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48264 (local address 127.18.70.65:40809)
0806 02:15:25.107510 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:25.107527 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:25.107574 (+ 47us) server_negotiation.cc:407] Connection header received
0806 02:15:25.107946 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:25.107954 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:25.108130 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:25.108296 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:25.108522 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.109262 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.110859 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.111743 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.111855 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.111998 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:25.112818 (+ 820us) server_negotiation.cc:299] Negotiation successful
0806 02:15:25.112908 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":971,"thread_start_us":89,"threads_started":1}
I20260806 02:15:25.115641 2809 tablet_service.cc:1467] Processing CreateTablet for tablet 02967dbbf78144f9b7a611017b276d7b (DEFAULT_TABLE table=test-table [id=33dc21ad4f9140cabc06c44462b33c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:15:25.115965 2518 tablet_service.cc:1467] Processing CreateTablet for tablet 02967dbbf78144f9b7a611017b276d7b (DEFAULT_TABLE table=test-table [id=33dc21ad4f9140cabc06c44462b33c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:15:25.116873 2809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02967dbbf78144f9b7a611017b276d7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:25.116868 2518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02967dbbf78144f9b7a611017b276d7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:25.124315 2899 tablet_bootstrap.cc:492] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf: Bootstrap starting.
I20260806 02:15:25.124365 2667 tablet_service.cc:1467] Processing CreateTablet for tablet 02967dbbf78144f9b7a611017b276d7b (DEFAULT_TABLE table=test-table [id=33dc21ad4f9140cabc06c44462b33c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:15:25.124796 2898 tablet_bootstrap.cc:492] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Bootstrap starting.
I20260806 02:15:25.125447 2667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02967dbbf78144f9b7a611017b276d7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:25.127137 2898 tablet_bootstrap.cc:654] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:25.128584 2898 log.cc:824] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:25.128937 2899 tablet_bootstrap.cc:654] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:25.130086 2899 log.cc:824] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:25.130822 2898 tablet_bootstrap.cc:492] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: No bootstrap required, opened a new log
I20260806 02:15:25.131026 2898 ts_tablet_manager.cc:1397] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260806 02:15:25.131866 2899 tablet_bootstrap.cc:492] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf: No bootstrap required, opened a new log
I20260806 02:15:25.132061 2899 ts_tablet_manager.cc:1397] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 02:15:25.132820 2902 tablet_bootstrap.cc:492] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611: Bootstrap starting.
I20260806 02:15:25.134495 2898 raft_consensus.cc:348] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } } peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } }
I20260806 02:15:25.134757 2898 raft_consensus.cc:374] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:25.134832 2898 raft_consensus.cc:729] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f7277bf61e4409080ccc39f9540a80f, State: Initialized, Role: FOLLOWER
I20260806 02:15:25.135344 2898 consensus_queue.cc:260] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } } peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } }
I20260806 02:15:25.135874 2902 tablet_bootstrap.cc:654] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:25.135809 2899 raft_consensus.cc:348] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } } peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } }
I20260806 02:15:25.136147 2899 raft_consensus.cc:374] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:25.136160 2590 heartbeater.cc:503] Master 127.18.70.126:35757 was elected leader, sending a full tablet report...
I20260806 02:15:25.136255 2899 raft_consensus.cc:729] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d502f01a8fd44cd08a55090c1958e1cf, State: Initialized, Role: FOLLOWER
I20260806 02:15:25.136518 2898 ts_tablet_manager.cc:1428] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260806 02:15:25.136811 2902 log.cc:824] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:25.136772 2899 consensus_queue.cc:260] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } } peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } }
I20260806 02:15:25.137425 2879 heartbeater.cc:503] Master 127.18.70.126:35757 was elected leader, sending a full tablet report...
I20260806 02:15:25.137727 2899 ts_tablet_manager.cc:1428] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:15:25.138746 2902 tablet_bootstrap.cc:492] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611: No bootstrap required, opened a new log
I20260806 02:15:25.138948 2902 ts_tablet_manager.cc:1397] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:15:25.142333 2902 raft_consensus.cc:348] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } } peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } }
I20260806 02:15:25.142570 2902 raft_consensus.cc:374] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:25.142627 2902 raft_consensus.cc:729] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fdec9444a874f5185a7ae5a8d3a5611, State: Initialized, Role: FOLLOWER
I20260806 02:15:25.142982 2902 consensus_queue.cc:260] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } } peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } }
I20260806 02:15:25.143628 2736 heartbeater.cc:503] Master 127.18.70.126:35757 was elected leader, sending a full tablet report...
I20260806 02:15:25.143975 2902 ts_tablet_manager.cc:1428] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 02:15:25.204615 2903 raft_consensus.cc:482] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:15:25.204820 2903 raft_consensus.cc:504] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } } peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } }
I20260806 02:15:25.206103 2903 leader_election.cc:288] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689), d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737)
I20260806 02:15:25.210139 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.206530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:40277 (local address 127.18.70.67:35737)
0806 02:15:25.206644 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:25.206649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:25.206822 (+ 173us) server_negotiation.cc:407] Connection header received
0806 02:15:25.206997 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:25.207000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:25.207040 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:25.207121 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:25.207427 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.207999 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.209132 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.209514 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.209641 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.209693 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:25.209937 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 02:15:25.210020 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 02:15:25.210160 2907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.206371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:35737 (local address 127.18.70.65:40277)
0806 02:15:25.206711 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:25.206722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:25.206867 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:25.207131 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:25.207133 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:25.207174 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:25.207313 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:25.207318 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.208121 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.208125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:25.208999 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:25.209008 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.209633 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.209637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:25.209730 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.209816 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 02:15:25.209919 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 02:15:25.210007 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":55,"spinlock_wait_cycles":2176,"thread_start_us":110,"threads_started":1}
I20260806 02:15:25.210155 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.206389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:41399 (local address 127.18.70.66:42689)
0806 02:15:25.206532 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:25.206537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:25.206759 (+ 222us) server_negotiation.cc:407] Connection header received
0806 02:15:25.206956 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:25.206964 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:25.207007 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:25.207093 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:25.207435 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.208004 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.209147 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.209534 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.209661 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.209715 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:25.209952 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 02:15:25.210039 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 02:15:25.210177 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.206273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:42689 (local address 127.18.70.65:41399)
0806 02:15:25.206644 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:25.206660 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:25.206821 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:25.207105 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:25.207108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:25.207123 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:25.207307 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:25.207312 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.208132 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.208137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:25.209017 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:25.209027 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.209654 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.209658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:25.209794 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.209831 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 02:15:25.209934 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 02:15:25.210025 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":86,"spinlock_wait_cycles":24448,"thread_start_us":120,"threads_started":1}
I20260806 02:15:25.210745 2687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02967dbbf78144f9b7a611017b276d7b" candidate_uuid: "9f7277bf61e4409080ccc39f9540a80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" is_pre_election: true
I20260806 02:15:25.210894 2829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02967dbbf78144f9b7a611017b276d7b" candidate_uuid: "9f7277bf61e4409080ccc39f9540a80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d502f01a8fd44cd08a55090c1958e1cf" is_pre_election: true
I20260806 02:15:25.211088 2687 raft_consensus.cc:2454] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f7277bf61e4409080ccc39f9540a80f in term 0.
I20260806 02:15:25.211205 2829 raft_consensus.cc:2454] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f7277bf61e4409080ccc39f9540a80f in term 0.
I20260806 02:15:25.211623 2463 leader_election.cc:302] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fdec9444a874f5185a7ae5a8d3a5611, 9f7277bf61e4409080ccc39f9540a80f; no voters:
I20260806 02:15:25.211891 2903 raft_consensus.cc:2781] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:15:25.211974 2903 raft_consensus.cc:482] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:15:25.212015 2903 raft_consensus.cc:3037] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:25.212981 2903 raft_consensus.cc:504] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } } peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } }
I20260806 02:15:25.213354 2903 leader_election.cc:288] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 1 election: Requested vote from peers 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689), d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737)
I20260806 02:15:25.213677 2687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02967dbbf78144f9b7a611017b276d7b" candidate_uuid: "9f7277bf61e4409080ccc39f9540a80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fdec9444a874f5185a7ae5a8d3a5611"
I20260806 02:15:25.213732 2829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02967dbbf78144f9b7a611017b276d7b" candidate_uuid: "9f7277bf61e4409080ccc39f9540a80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d502f01a8fd44cd08a55090c1958e1cf"
I20260806 02:15:25.213793 2687 raft_consensus.cc:3037] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:25.213850 2829 raft_consensus.cc:3037] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:25.214874 2687 raft_consensus.cc:2454] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f7277bf61e4409080ccc39f9540a80f in term 1.
I20260806 02:15:25.214948 2829 raft_consensus.cc:2454] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f7277bf61e4409080ccc39f9540a80f in term 1.
I20260806 02:15:25.215183 2463 leader_election.cc:302] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fdec9444a874f5185a7ae5a8d3a5611, 9f7277bf61e4409080ccc39f9540a80f; no voters:
I20260806 02:15:25.215394 2903 raft_consensus.cc:2781] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:15:25.215682 2903 raft_consensus.cc:686] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 1 LEADER]: Becoming Leader. State: Replica: 9f7277bf61e4409080ccc39f9540a80f, State: Running, Role: LEADER
I20260806 02:15:25.216013 2903 consensus_queue.cc:237] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } } peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } }
I20260806 02:15:25.219340 2377 catalog_manager.cc:5495] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f7277bf61e4409080ccc39f9540a80f (127.18.70.65). New cstate: current_term: 1 leader_uuid: "9f7277bf61e4409080ccc39f9540a80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } health_report { overall_health: UNKNOWN } } }
I20260806 02:15:25.232996 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.229434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48268 (local address 127.18.70.65:40809)
0806 02:15:25.229570 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:25.229575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:25.229589 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:15:25.229770 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:25.229772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:25.229828 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:25.229935 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:15:25.230179 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.230649 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.231539 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:25.231795 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:25.231904 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:25.232219 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:15:25.232612 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:15:25.232746 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:25.232818 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 02:15:25.232867 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 02:15:25.238448 2829 raft_consensus.cc:1261] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf [term 1 FOLLOWER]: Refusing update from remote peer 9f7277bf61e4409080ccc39f9540a80f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:15:25.238447 2687 raft_consensus.cc:1261] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611 [term 1 FOLLOWER]: Refusing update from remote peer 9f7277bf61e4409080ccc39f9540a80f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:15:25.239156 2910 consensus_queue.cc:1035] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:15:25.239323 2903 consensus_queue.cc:1035] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:15:25.246940 2914 mvcc.cc:205] Tried to move back new op lower bound from 7315384423370014720 to 7315384423287472128. Current Snapshot: MvccSnapshot[applied={T|T < 7315384423370014720}]
I20260806 02:15:25.249768 2915 mvcc.cc:205] Tried to move back new op lower bound from 7315384423370014720 to 7315384423287472128. Current Snapshot: MvccSnapshot[applied={T|T < 7315384423370014720}]
I20260806 02:15:25.254832 2377 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:15:25.254966 2377 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:15:25.257292 2377 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Sending DeleteTablet for 3 replicas of tablet 02967dbbf78144f9b7a611017b276d7b
I20260806 02:15:25.258030 2518 tablet_service.cc:1514] Processing DeleteTablet for tablet 02967dbbf78144f9b7a611017b276d7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:15:25 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:48264
I20260806 02:15:25.258062 2667 tablet_service.cc:1514] Processing DeleteTablet for tablet 02967dbbf78144f9b7a611017b276d7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:15:25 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:42436
I20260806 02:15:25.258502 2809 tablet_service.cc:1514] Processing DeleteTablet for tablet 02967dbbf78144f9b7a611017b276d7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:15:25 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:43446
I20260806 02:15:25.258577 2924 tablet_replica.cc:330] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: stopping tablet replica
I20260806 02:15:25.258869 2924 raft_consensus.cc:2229] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:15:25.258949 2926 tablet_replica.cc:330] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf: stopping tablet replica
I20260806 02:15:25.259019 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 2446
I20260806 02:15:25.260771 2926 raft_consensus.cc:2229] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:15:25.261134 2926 raft_consensus.cc:2258] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:15:25.262483 2926 ts_tablet_manager.cc:1905] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:15:25.263734 2925 tablet_replica.cc:330] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611: stopping tablet replica
I20260806 02:15:25.263998 2925 raft_consensus.cc:2229] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:15:25.264282 2925 raft_consensus.cc:2258] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:15:25.265200 2926 ts_tablet_manager.cc:1918] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:15:25.265309 2926 log.cc:1196] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/02967dbbf78144f9b7a611017b276d7b
I20260806 02:15:25.265638 2926 ts_tablet_manager.cc:1939] T 02967dbbf78144f9b7a611017b276d7b P d502f01a8fd44cd08a55090c1958e1cf: Deleting consensus metadata
I20260806 02:15:25.265705 2925 ts_tablet_manager.cc:1905] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:15:25.266942 2364 catalog_manager.cc:4841] TS d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737): tablet 02967dbbf78144f9b7a611017b276d7b (table test-table [id=33dc21ad4f9140cabc06c44462b33c41]) successfully deleted
I20260806 02:15:25.268047 2925 ts_tablet_manager.cc:1918] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:15:25.268157 2925 log.cc:1196] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/02967dbbf78144f9b7a611017b276d7b
I20260806 02:15:25.268537 2925 ts_tablet_manager.cc:1939] T 02967dbbf78144f9b7a611017b276d7b P 6fdec9444a874f5185a7ae5a8d3a5611: Deleting consensus metadata
I20260806 02:15:25.269553 2360 catalog_manager.cc:4841] TS 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689): tablet 02967dbbf78144f9b7a611017b276d7b (table test-table [id=33dc21ad4f9140cabc06c44462b33c41]) successfully deleted
W20260806 02:15:25.269707 2363 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:40809 (error 108)
I20260806 02:15:25.269845 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 2596
I20260806 02:15:25.272027 2892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:25.271546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:40809 (local address 127.0.0.1:48272)
0806 02:15:25.271823 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:25.271917 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:40809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
W20260806 02:15:25.272197 2363 catalog_manager.cc:4584] TS 9f7277bf61e4409080ccc39f9540a80f (127.18.70.65:40809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 02967dbbf78144f9b7a611017b276d7b: Network error: Client connection negotiation failed: client connection to 127.18.70.65:40809: connect: Connection refused (error 111)
I20260806 02:15:25.277568 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 2741
I20260806 02:15:25.284524 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 2340
I20260806 02:15:45.293349 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:35757
--webserver_interface=127.18.70.126
--webserver_port=40031
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:35757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:15:45.403995 2928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:45.404280 2928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:45.404358 2928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:45.408294 2928 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 02:15:45.408365 2928 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 02:15:45.408394 2928 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 02:15:45.408427 2928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:15:45.408463 2928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:45.408522 2928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:45.408558 2928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:15:45.408591 2928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:15:45.413726 2928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:35757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:35757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=40031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:45.414776 2928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:45.415807 2928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:45.430020 2938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:45.430028 2935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:45.430159 2928 server_base.cc:1034] running on GCE node
W20260806 02:15:45.430027 2936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:45.430955 2928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:45.432104 2928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:45.433264 2928 hybrid_clock.cc:648] HybridClock initialized: now 1785982545433253 us; error 33 us; skew 500 ppm
Aug 06 02:15:45 dist-test-slave-6k22 krb5kdc[2326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982545, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:45.441222 2928 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:15:45.442344 2928 webserver.cc:460] Webserver started at http://127.18.70.126:40031/ using document root <none> and password file <none>
I20260806 02:15:45.442946 2928 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:45.443084 2928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:45.447000 2928 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 02:15:45.449247 2946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:45.450564 2928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:15:45.450734 2928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "6a71232f5f23d82bb6eae54d0e764ce2"
server_key_iv: "c07e18b46fac918e8b70c4924429ffd4"
server_key_version: "encryptionkey@0"
I20260806 02:15:45.451352 2928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:45.472791 2928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:45.475893 2928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:45.476126 2928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:45.483611 2928 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:35757
I20260806 02:15:45.483898 3003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:35757 every 8 connection(s)
I20260806 02:15:45.484787 2928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:15:45.489154 3004 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:15:45.490844 3004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Bootstrap starting.
I20260806 02:15:45.490944 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 2928
I20260806 02:15:45.491451 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:40809
--local_ip_for_outbound_sockets=127.18.70.65
--tserver_master_addrs=127.18.70.126:35757
--webserver_port=35553
--webserver_interface=127.18.70.65
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:15:45.508370 3004 log.cc:824] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:45.517122 3004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:15:45.517555 3004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Bootstrap complete.
I20260806 02:15:45.521847 3004 raft_consensus.cc:348] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } }
I20260806 02:15:45.522251 3004 raft_consensus.cc:729] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e370ecdf144c4b5facc77b7ecbc9c2cc, State: Initialized, Role: FOLLOWER
I20260806 02:15:45.522779 3004 consensus_queue.cc:260] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } }
I20260806 02:15:45.522943 3004 raft_consensus.cc:388] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:15:45.523031 3004 raft_consensus.cc:482] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:15:45.523146 3004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 1 FOLLOWER]: Advancing to term 2
I20260806 02:15:45.525113 3004 raft_consensus.cc:504] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } }
I20260806 02:15:45.525458 3004 leader_election.cc:302] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e370ecdf144c4b5facc77b7ecbc9c2cc; no voters:
I20260806 02:15:45.525782 3004 leader_election.cc:288] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 02:15:45.525918 3008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 2 FOLLOWER]: Leader election won for term 2
I20260806 02:15:45.526163 3008 raft_consensus.cc:686] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 2 LEADER]: Becoming Leader. State: Replica: e370ecdf144c4b5facc77b7ecbc9c2cc, State: Running, Role: LEADER
I20260806 02:15:45.526610 3008 consensus_queue.cc:237] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } }
I20260806 02:15:45.527042 3004 sys_catalog.cc:564] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:15:45.528172 3010 sys_catalog.cc:455] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader e370ecdf144c4b5facc77b7ecbc9c2cc. Latest consensus state: current_term: 2 leader_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } } }
I20260806 02:15:45.528331 3010 sys_catalog.cc:458] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:45.528594 3009 sys_catalog.cc:455] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } } }
I20260806 02:15:45.528674 3009 sys_catalog.cc:458] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: This master's current role is: LEADER
I20260806 02:15:45.528729 3017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:15:45.533689 3017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:15:45.534423 3017 catalog_manager.cc:1259] Loaded cluster ID: c8298e54946c484580cd31c97823b030
I20260806 02:15:45.534488 3017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:15:45.536973 3017 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:15:45.537694 3017 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Loaded TSK: 0
I20260806 02:15:45.552009 3017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Generated new TSK 1
I20260806 02:15:45.552279 3017 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 02:15:45.625483 3006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:45.625897 3006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:45.625998 3006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:45.632568 3006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:45.632642 3006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:45.632740 3006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:15:45.639186 3006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:40809
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=35553
--tserver_master_addrs=127.18.70.126:35757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:45.640352 3006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:45.641461 3006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:45.652253 3032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:45.652266 3034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:45.652552 3031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:45.653362 3006 server_base.cc:1034] running on GCE node
I20260806 02:15:45.653762 3006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:45.654471 3006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:45.655637 3006 hybrid_clock.cc:648] HybridClock initialized: now 1785982545655614 us; error 31 us; skew 500 ppm
Aug 06 02:15:45 dist-test-slave-6k22 krb5kdc[2326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982545, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:45.658524 3006 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:15:45.659408 3006 webserver.cc:460] Webserver started at http://127.18.70.65:35553/ using document root <none> and password file <none>
I20260806 02:15:45.660043 3006 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:45.660144 3006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:45.664105 3006 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:15:45.666340 3043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:45.667752 3006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:15:45.667893 3006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "9f7277bf61e4409080ccc39f9540a80f"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "5213d070314541d6d147277d1e58ed77"
server_key_iv: "f18ce112d48718eab89c9c92f329ca99"
server_key_version: "encryptionkey@0"
I20260806 02:15:45.668438 3006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:45.710218 3006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:45.713414 3006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:45.713639 3006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:45.714308 3006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:45.715432 3050 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 02:15:45.719789 3006 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 02:15:45.719866 3006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260806 02:15:45.719949 3006 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 02:15:45.721632 3006 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 02:15:45.721697 3006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 02:15:45.721966 3050 tablet_bootstrap.cc:492] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Bootstrap starting.
I20260806 02:15:45.732357 3006 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:40809
I20260806 02:15:45.732533 3169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:40809 every 8 connection(s)
I20260806 02:15:45.733341 3006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:15:45.739636 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 3006
I20260806 02:15:45.740077 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:42689
--local_ip_for_outbound_sockets=127.18.70.66
--tserver_master_addrs=127.18.70.126:35757
--webserver_port=34571
--webserver_interface=127.18.70.66
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:15:45.741523 3050 log.cc:824] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Log is configured to *not* fsync() on all Append() calls
Aug 06 02:15:45 dist-test-slave-6k22 krb5kdc[2326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982545, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:45.749936 3050 tablet_bootstrap.cc:492] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:15:45.750334 3050 tablet_bootstrap.cc:492] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Bootstrap complete.
I20260806 02:15:45.751619 3050 ts_tablet_manager.cc:1397] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Time spent bootstrapping tablet: real 0.030s user 0.028s sys 0.001s
I20260806 02:15:45.752863 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:45.735524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:39293 (local address 127.18.70.126:35757)
0806 02:15:45.735949 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:45.735978 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:45.736276 (+ 298us) server_negotiation.cc:407] Connection header received
0806 02:15:45.737401 (+ 1125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:45.737421 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:45.737957 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:45.738375 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:45.739391 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:45.740856 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:45.742607 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:45.743300 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:45.743467 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:45.746439 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:45.746446 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:45.746462 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:45.746496 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:45.748722 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:45.749440 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:45.749446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:45.749452 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:45.749602 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:45.750067 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:45.750072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:45.750077 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:45.750614 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:45.750747 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:45.751583 (+ 836us) server_negotiation.cc:299] Negotiation successful
0806 02:15:45.751991 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":122,"threads_started":1}
I20260806 02:15:45.752914 3173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:45.735446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:39293)
0806 02:15:45.736089 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:45.736141 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:45.736943 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:45.738419 (+ 1476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:45.738425 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:45.738753 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:45.739201 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:45.739214 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:45.741031 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:45.741037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:45.742460 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:45.742474 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:45.743452 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:45.743458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:45.743674 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:45.744307 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:45.744339 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:45.746222 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:45.748860 (+ 2638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:45.748869 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:45.748884 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:45.749264 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:45.749718 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:45.749724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:45.749729 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:45.749930 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:45.750848 (+ 918us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:45.750856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:45.751175 (+ 319us) client_negotiation.cc:769] Sending connection context
0806 02:15:45.751408 (+ 233us) client_negotiation.cc:240] Negotiation successful
0806 02:15:45.751814 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":151,"threads_started":1}
I20260806 02:15:45.754540 3050 raft_consensus.cc:348] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } } peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } }
I20260806 02:15:45.754943 3050 raft_consensus.cc:729] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f7277bf61e4409080ccc39f9540a80f, State: Initialized, Role: FOLLOWER
I20260806 02:15:45.755481 3050 consensus_queue.cc:260] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } } peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } }
I20260806 02:15:45.755674 3170 heartbeater.cc:347] Connected to a master server at 127.18.70.126:35757
I20260806 02:15:45.755914 3170 heartbeater.cc:464] Registering TS with master...
I20260806 02:15:45.756475 3170 heartbeater.cc:511] Master 127.18.70.126:35757 requested a full tablet report, sending...
I20260806 02:15:45.756551 3050 ts_tablet_manager.cc:1428] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 02:15:45.759024 2965 ts_manager.cc:194] Registered new tserver with Master: 9f7277bf61e4409080ccc39f9540a80f (127.18.70.65:40809)
I20260806 02:15:45.763008 2965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:39293
I20260806 02:15:45.765504 3170 heartbeater.cc:503] Master 127.18.70.126:35757 was elected leader, sending a full tablet report...
Aug 06 02:15:45 dist-test-slave-6k22 krb5kdc[2326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982545, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for kudu/127.18.70.65@KRBTEST.COM
I20260806 02:15:45.773454 3177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:45.762337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:40809 (local address 127.0.0.1:53430)
0806 02:15:45.763014 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:45.763047 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:45.763251 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:45.763935 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:45.763943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:45.764400 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:45.764527 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:45.764535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:45.765948 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:45.765951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:45.766620 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:45.766630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:45.767420 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:45.767424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:45.767545 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:45.768119 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:45.768131 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:45.769670 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:45.771554 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:45.771584 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:45.771590 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:45.771900 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:45.772211 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:45.772214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:45.772216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:45.772324 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:45.772815 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:45.772822 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:45.772950 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 02:15:45.773093 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 02:15:45.773281 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":89,"threads_started":1}
I20260806 02:15:45.773496 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:45.762407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53430 (local address 127.18.70.65:40809)
0806 02:15:45.762780 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:45.762791 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:45.763143 (+ 352us) server_negotiation.cc:407] Connection header received
0806 02:15:45.763419 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:45.763424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:45.763671 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:45.763832 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:45.764663 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:45.765840 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:45.766749 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:45.767281 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:45.767407 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:45.769833 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:45.769843 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:45.769856 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:45.769881 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:45.771437 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:45.772024 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:45.772030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:45.772032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:45.772116 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:45.772445 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:45.772450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:45.772452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:45.772686 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:45.772850 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:45.773177 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 02:15:45.773348 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":289,"thread_start_us":99,"threads_started":1}
I20260806 02:15:45.774389 3094 tablet_service.cc:1514] Processing DeleteTablet for tablet 02967dbbf78144f9b7a611017b276d7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:15:25 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:53430
I20260806 02:15:45.775197 3179 tablet_replica.cc:330] stopping tablet replica
I20260806 02:15:45.775484 3179 raft_consensus.cc:2229] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:15:45.775755 3179 raft_consensus.cc:2258] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:15:45.776737 3179 ts_tablet_manager.cc:1905] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:15:45.780397 3179 ts_tablet_manager.cc:1918] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:15:45.780476 3179 log.cc:1196] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal/wals/02967dbbf78144f9b7a611017b276d7b
I20260806 02:15:45.780766 3179 ts_tablet_manager.cc:1939] T 02967dbbf78144f9b7a611017b276d7b P 9f7277bf61e4409080ccc39f9540a80f: Deleting consensus metadata
I20260806 02:15:45.782369 2949 catalog_manager.cc:4841] TS 9f7277bf61e4409080ccc39f9540a80f (127.18.70.65:40809): tablet 02967dbbf78144f9b7a611017b276d7b (table test-table [id=33dc21ad4f9140cabc06c44462b33c41]) successfully deleted
W20260806 02:15:45.861780 3174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:45.862040 3174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:45.862108 3174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:45.865998 3174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:45.866073 3174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:45.866173 3174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:15:45.871150 3174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:42689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=34571
--tserver_master_addrs=127.18.70.126:35757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:45.872253 3174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:45.873322 3174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:45.882844 3184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:45.882884 3185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:45.883046 3174 server_base.cc:1034] running on GCE node
W20260806 02:15:45.882906 3187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:45.883524 3174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:45.884276 3174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:45.885435 3174 hybrid_clock.cc:648] HybridClock initialized: now 1785982545885386 us; error 58 us; skew 500 ppm
Aug 06 02:15:45 dist-test-slave-6k22 krb5kdc[2326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982545, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:45.888185 3174 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:15:45.889043 3174 webserver.cc:460] Webserver started at http://127.18.70.66:34571/ using document root <none> and password file <none>
I20260806 02:15:45.889576 3174 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:45.889683 3174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:45.893329 3174 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:15:45.895310 3196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:45.896498 3174 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:15:45.896628 3174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "6fdec9444a874f5185a7ae5a8d3a5611"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "e85c266fc46503137064412a508f4343"
server_key_iv: "27a9d48d6c579fd9323206893489e047"
server_key_version: "encryptionkey@0"
I20260806 02:15:45.897137 3174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:45.916224 3174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:45.919343 3174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:45.919544 3174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:45.920135 3174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:45.920950 3174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:15:45.921020 3174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:45.921089 3174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:15:45.921132 3174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:45.930141 3174 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:42689
I20260806 02:15:45.930171 3320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:42689 every 8 connection(s)
I20260806 02:15:45.931082 3174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:15:45.939498 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 3174
I20260806 02:15:45.939951 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:35737
--local_ip_for_outbound_sockets=127.18.70.67
--tserver_master_addrs=127.18.70.126:35757
--webserver_port=38177
--webserver_interface=127.18.70.67
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 02:15:45 dist-test-slave-6k22 krb5kdc[2326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982545, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:45.944368 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:45.932810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:34521 (local address 127.18.70.126:35757)
0806 02:15:45.932948 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:45.932952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:45.934200 (+ 1248us) server_negotiation.cc:407] Connection header received
0806 02:15:45.935004 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:45.935007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:45.935055 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:45.935131 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:45.936266 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:45.936788 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:45.937852 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:45.938067 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:45.938140 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:45.940643 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:45.940651 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:45.940657 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:45.940691 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:45.942238 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:45.942851 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:45.942854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:45.942855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:45.942936 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:45.943306 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:45.943308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:45.943309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:45.943461 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:45.943526 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:45.944138 (+ 612us) server_negotiation.cc:299] Negotiation successful
0806 02:15:45.944248 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 02:15:45.945559 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:45.933166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:34521)
0806 02:15:45.934050 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:45.934091 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:45.934821 (+ 730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:45.935293 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:45.935299 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:45.935661 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:45.936135 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:45.936148 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:45.936896 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:45.936898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:45.937738 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:45.937747 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:45.938192 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:45.938195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:45.938335 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:45.938894 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:45.938911 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:45.940469 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:45.942367 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:45.942374 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:45.942390 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:45.942744 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:45.943021 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:45.943026 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:45.943029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:45.943207 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:45.943580 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:45.943589 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:45.943875 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 02:15:45.944070 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 02:15:45.944442 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":551,"threads_started":1}
I20260806 02:15:45.946779 3321 heartbeater.cc:347] Connected to a master server at 127.18.70.126:35757
I20260806 02:15:45.947054 3321 heartbeater.cc:464] Registering TS with master...
I20260806 02:15:45.947721 3321 heartbeater.cc:511] Master 127.18.70.126:35757 requested a full tablet report, sending...
I20260806 02:15:45.948911 2965 ts_manager.cc:194] Registered new tserver with Master: 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689)
I20260806 02:15:45.949477 2965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:34521
W20260806 02:15:46.055111 3324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:15:46.055384 3324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:15:46.055454 3324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:15:46.059516 3324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:15:46.059700 3324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:15:46.059774 3324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:15:46.064741 3324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:35737
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=38177
--tserver_master_addrs=127.18.70.126:35757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:15:46.065642 3324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:15:46.066598 3324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:15:46.076244 3331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:46.076361 3324 server_base.cc:1034] running on GCE node
W20260806 02:15:46.076253 3333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:15:46.076262 3330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:15:46.076896 3324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:15:46.077550 3324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:15:46.078763 3324 hybrid_clock.cc:648] HybridClock initialized: now 1785982546078743 us; error 26 us; skew 500 ppm
Aug 06 02:15:46 dist-test-slave-6k22 krb5kdc[2326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982546, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:15:46.081568 3324 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:15:46.082405 3324 webserver.cc:460] Webserver started at http://127.18.70.67:38177/ using document root <none> and password file <none>
I20260806 02:15:46.082971 3324 fs_manager.cc:359] Metadata directory not provided
I20260806 02:15:46.083045 3324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:15:46.086805 3324 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:15:46.088856 3340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:46.090058 3324 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:15:46.090194 3324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "d502f01a8fd44cd08a55090c1958e1cf"
format_stamp: "Formatted at 2026-08-06 02:15:25 on dist-test-slave-6k22"
server_key: "0483c2fd2ad70904616567362629dbf0"
server_key_iv: "e79609d1f2111060f6186231de428dac"
server_key_version: "encryptionkey@0"
I20260806 02:15:46.090711 3324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:15:46.101603 3324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:15:46.104599 3324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:15:46.104796 3324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:15:46.105370 3324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:15:46.106189 3324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:15:46.106236 3324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:46.106302 3324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:15:46.106348 3324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:15:46.115340 3324 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:35737
I20260806 02:15:46.115360 3461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:35737 every 8 connection(s)
I20260806 02:15:46.116420 3324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:15:46.117702 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 3324
I20260806 02:15:46.126945 3465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.118837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37964 (local address 127.18.70.126:35757)
0806 02:15:46.119052 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:46.119056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:46.119068 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:15:46.119110 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:46.119113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:46.119151 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:46.119232 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:46.120174 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.121052 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.121854 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.122193 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.122334 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.123699 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:46.123703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:46.123706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:46.123734 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:46.125236 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:46.125608 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:46.125614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:46.125618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:46.125703 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:46.126029 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:46.126034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:46.126036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:46.126282 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:46.126513 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:46.126652 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 02:15:46.126767 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"spinlock_wait_cycles":896,"thread_start_us":73,"threads_started":1}
Aug 06 02:15:46 dist-test-slave-6k22 krb5kdc[2326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982546, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:15:46.131251 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.118241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:58799 (local address 127.18.70.126:35757)
0806 02:15:46.118437 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:46.118442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:46.119720 (+ 1278us) server_negotiation.cc:407] Connection header received
0806 02:15:46.121341 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:46.121344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:46.121390 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:46.121464 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:46.122791 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.123292 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.124890 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.125079 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.125149 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.127996 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:46.127999 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:46.128001 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:46.128022 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:46.129471 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:46.129984 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:46.129986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:46.129987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:46.130058 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:46.130357 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:46.130359 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:46.130360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:46.130524 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:46.130595 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:46.131061 (+ 466us) server_negotiation.cc:299] Negotiation successful
0806 02:15:46.131145 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66,"spinlock_wait_cycles":28800}
I20260806 02:15:46.132078 3466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.118722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:58799)
0806 02:15:46.119527 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:46.119620 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:46.121130 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:46.121632 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:46.121639 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:46.122063 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:15:46.122647 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:46.122659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.123415 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.123420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:46.124758 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:46.124771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.125226 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.125230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:46.125420 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.125914 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:15:46.125929 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:15:46.127813 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:15:46.129592 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:46.129596 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:46.129607 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:46.129887 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:46.130134 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:15:46.130136 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:15:46.130137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:15:46.130273 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:15:46.130609 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:15:46.130614 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:15:46.130857 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 02:15:46.130997 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 02:15:46.131223 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":133,"threads_started":1}
I20260806 02:15:46.133044 3462 heartbeater.cc:347] Connected to a master server at 127.18.70.126:35757
I20260806 02:15:46.133281 3462 heartbeater.cc:464] Registering TS with master...
I20260806 02:15:46.133741 3462 heartbeater.cc:511] Master 127.18.70.126:35757 requested a full tablet report, sending...
I20260806 02:15:46.134724 2965 ts_manager.cc:194] Registered new tserver with Master: d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737)
I20260806 02:15:46.135264 2965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:58799
I20260806 02:15:46.140519 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:15:46.144161 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.141105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37972 (local address 127.18.70.126:35757)
0806 02:15:46.141224 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:46.141228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:46.141416 (+ 188us) server_negotiation.cc:407] Connection header received
0806 02:15:46.141530 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:46.141532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:46.141615 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:46.141693 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:15:46.141892 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.142348 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.143156 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.143346 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.143414 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.143758 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:15:46.143870 (+ 112us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 02:15:46.144039 (+ 169us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37972: token expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 02:15:46.144232 3172 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37972: token expired
W20260806 02:15:46.144610 2886 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.70.126:35757)
I20260806 02:15:46.151113 3172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.144925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37986 (local address 127.18.70.126:35757)
0806 02:15:46.145036 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:46.145039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:46.145051 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:15:46.145136 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:46.145138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:46.145170 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:46.145232 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:15:46.145783 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.146236 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.146962 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.147150 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.147220 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.148415 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:15:46.148418 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:15:46.148420 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:15:46.148440 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:15:46.149821 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:46.150165 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:46.150168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:46.150169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:46.150235 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:15:46.150445 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:15:46.150447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:15:46.150448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:15:46.150589 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:15:46.150683 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:46.150909 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 02:15:46.151009 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 02:15:46.153324 2965 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:15:46.156558 2965 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:15:46.164206 3094 tablet_service.cc:1467] Processing CreateTablet for tablet 8a30d673d56b4d2a9c39f7c8ca30a2d1 (DEFAULT_TABLE table=test-table [id=73733fb772e04311995f95392a25c972]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:15:46.164914 3094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a30d673d56b4d2a9c39f7c8ca30a2d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:46.167794 3050 tablet_bootstrap.cc:492] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f: Bootstrap starting.
I20260806 02:15:46.167889 3177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.162837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:42689 (local address 127.0.0.1:36704)
0806 02:15:46.162934 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:46.162948 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:46.163018 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:46.163986 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:46.163989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:46.164006 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:46.164109 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:46.164116 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.165552 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.165557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:46.166480 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:46.166490 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.167404 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.167408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:46.167521 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.167594 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 02:15:46.167676 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 02:15:46.167759 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 02:15:46.168325 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.163034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36704 (local address 127.18.70.66:42689)
0806 02:15:46.163393 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:46.163405 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:46.163430 (+ 25us) server_negotiation.cc:407] Connection header received
0806 02:15:46.163510 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:46.163518 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:46.163773 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:46.164007 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:46.164208 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.165427 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.166614 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.167267 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.167381 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.167517 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:46.168052 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 02:15:46.168149 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":167,"threads_started":1}
I20260806 02:15:46.168637 3050 tablet_bootstrap.cc:654] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:46.168895 3474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.163140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:35737 (local address 127.0.0.1:48274)
0806 02:15:46.163694 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:46.163707 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:46.163791 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:46.164376 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:46.164379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:46.164507 (+ 128us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:46.164593 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:46.164600 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.165844 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.165848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:46.166953 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:46.166964 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.168523 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.168527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:46.168608 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.168624 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 02:15:46.168704 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 02:15:46.168803 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":100,"threads_started":1}
I20260806 02:15:46.169395 3475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.163433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48274 (local address 127.18.70.67:35737)
0806 02:15:46.163898 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:46.163910 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:46.163932 (+ 22us) server_negotiation.cc:407] Connection header received
0806 02:15:46.164003 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:46.164008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:46.164180 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:46.164402 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:46.164724 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.165710 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.167096 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.168208 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.168335 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.169151 (+ 816us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:46.169210 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 02:15:46.169279 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":172,"threads_started":1}
I20260806 02:15:46.170943 3050 tablet_bootstrap.cc:492] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f: No bootstrap required, opened a new log
I20260806 02:15:46.171036 3050 ts_tablet_manager.cc:1397] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 02:15:46.171481 3050 raft_consensus.cc:348] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:15:46.171626 3050 raft_consensus.cc:374] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:46.171666 3050 raft_consensus.cc:729] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f7277bf61e4409080ccc39f9540a80f, State: Initialized, Role: FOLLOWER
I20260806 02:15:46.171808 3050 consensus_queue.cc:260] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:15:46.172420 3392 tablet_service.cc:1467] Processing CreateTablet for tablet 8a30d673d56b4d2a9c39f7c8ca30a2d1 (DEFAULT_TABLE table=test-table [id=73733fb772e04311995f95392a25c972]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:15:46.172420 3247 tablet_service.cc:1467] Processing CreateTablet for tablet 8a30d673d56b4d2a9c39f7c8ca30a2d1 (DEFAULT_TABLE table=test-table [id=73733fb772e04311995f95392a25c972]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:15:46.173266 3050 ts_tablet_manager.cc:1428] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 02:15:46.173542 3247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a30d673d56b4d2a9c39f7c8ca30a2d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:46.173578 3392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a30d673d56b4d2a9c39f7c8ca30a2d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:15:46.181171 3477 tablet_bootstrap.cc:492] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611: Bootstrap starting.
I20260806 02:15:46.182111 3478 tablet_bootstrap.cc:492] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf: Bootstrap starting.
I20260806 02:15:46.184973 3478 tablet_bootstrap.cc:654] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:46.184959 3477 tablet_bootstrap.cc:654] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611: Neither blocks nor log segments found. Creating new log.
I20260806 02:15:46.185907 3477 log.cc:824] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:46.185907 3478 log.cc:824] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf: Log is configured to *not* fsync() on all Append() calls
I20260806 02:15:46.187960 3478 tablet_bootstrap.cc:492] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf: No bootstrap required, opened a new log
I20260806 02:15:46.188122 3477 tablet_bootstrap.cc:492] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611: No bootstrap required, opened a new log
I20260806 02:15:46.188153 3478 ts_tablet_manager.cc:1397] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:15:46.188318 3477 ts_tablet_manager.cc:1397] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260806 02:15:46.190959 3477 raft_consensus.cc:348] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:15:46.190959 3478 raft_consensus.cc:348] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:15:46.191187 3478 raft_consensus.cc:374] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:46.191195 3477 raft_consensus.cc:374] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:15:46.191267 3478 raft_consensus.cc:729] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d502f01a8fd44cd08a55090c1958e1cf, State: Initialized, Role: FOLLOWER
I20260806 02:15:46.191293 3477 raft_consensus.cc:729] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fdec9444a874f5185a7ae5a8d3a5611, State: Initialized, Role: FOLLOWER
I20260806 02:15:46.191730 3477 consensus_queue.cc:260] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:15:46.191730 3478 consensus_queue.cc:260] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:15:46.192373 3321 heartbeater.cc:503] Master 127.18.70.126:35757 was elected leader, sending a full tablet report...
I20260806 02:15:46.192399 3462 heartbeater.cc:503] Master 127.18.70.126:35757 was elected leader, sending a full tablet report...
I20260806 02:15:46.192684 3477 ts_tablet_manager.cc:1428] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260806 02:15:46.193732 3478 ts_tablet_manager.cc:1428] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260806 02:15:46.234512 3171 tablet.cc:2369] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 02:15:46.367452 3463 tablet.cc:2369] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:15:46.384255 3481 raft_consensus.cc:482] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:15:46.384564 3481 raft_consensus.cc:504] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:15:46.386253 3481 leader_election.cc:288] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737), 9f7277bf61e4409080ccc39f9540a80f (127.18.70.65:40809)
I20260806 02:15:46.390950 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.386509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:35737 (local address 127.18.70.66:51015)
0806 02:15:46.386835 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:46.386851 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:46.386966 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:46.387622 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:46.387625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:46.387642 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:46.387776 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:46.387783 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.388504 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.388510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:46.389847 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:46.389861 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.390487 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.390491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:46.390584 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.390605 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 02:15:46.390710 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 02:15:46.390796 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":61,"spinlock_wait_cycles":23296}
I20260806 02:15:46.390942 3475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.387143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:51015 (local address 127.18.70.67:35737)
0806 02:15:46.387317 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:46.387324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:46.387343 (+ 19us) server_negotiation.cc:407] Connection header received
0806 02:15:46.387409 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:46.387414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:46.387475 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:46.387579 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:46.387875 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.388406 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.389974 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.390371 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.390491 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.390541 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:46.390730 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 02:15:46.390812 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 02:15:46.391011 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.386643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:51423 (local address 127.18.70.65:40809)
0806 02:15:46.386929 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:46.386935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:46.386957 (+ 22us) server_negotiation.cc:407] Connection header received
0806 02:15:46.387068 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:46.387073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:46.387150 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:46.387259 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:15:46.387815 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.388584 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.390025 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.390478 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.390602 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.390760 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:46.390829 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 02:15:46.390900 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":85,"threads_started":1}
I20260806 02:15:46.391021 3483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.386541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:40809 (local address 127.18.70.66:51423)
0806 02:15:46.386835 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:46.386851 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:15:46.386966 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:15:46.387398 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:15:46.387403 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:15:46.387426 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:15:46.387639 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:46.387653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.388754 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.388757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:46.389854 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:15:46.389885 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.390602 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.390606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:15:46.390687 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.390707 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 02:15:46.390769 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 02:15:46.390857 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":3,"spinlock_wait_cycles":8960,"thread_start_us":113,"threads_started":1}
I20260806 02:15:46.391709 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a30d673d56b4d2a9c39f7c8ca30a2d1" candidate_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f7277bf61e4409080ccc39f9540a80f" is_pre_election: true
I20260806 02:15:46.391865 3412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a30d673d56b4d2a9c39f7c8ca30a2d1" candidate_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d502f01a8fd44cd08a55090c1958e1cf" is_pre_election: true
I20260806 02:15:46.392030 3116 raft_consensus.cc:2454] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fdec9444a874f5185a7ae5a8d3a5611 in term 0.
I20260806 02:15:46.392103 3412 raft_consensus.cc:2454] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fdec9444a874f5185a7ae5a8d3a5611 in term 0.
I20260806 02:15:46.392544 3199 leader_election.cc:302] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6fdec9444a874f5185a7ae5a8d3a5611, 9f7277bf61e4409080ccc39f9540a80f, d502f01a8fd44cd08a55090c1958e1cf; no voters:
I20260806 02:15:46.392812 3481 raft_consensus.cc:2781] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:15:46.392910 3481 raft_consensus.cc:482] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:15:46.392962 3481 raft_consensus.cc:3037] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:46.394114 3481 raft_consensus.cc:504] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:15:46.394452 3481 leader_election.cc:288] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [CANDIDATE]: Term 1 election: Requested vote from peers d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737), 9f7277bf61e4409080ccc39f9540a80f (127.18.70.65:40809)
I20260806 02:15:46.394848 3412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a30d673d56b4d2a9c39f7c8ca30a2d1" candidate_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d502f01a8fd44cd08a55090c1958e1cf"
I20260806 02:15:46.394980 3412 raft_consensus.cc:3037] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:46.396044 3412 raft_consensus.cc:2454] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fdec9444a874f5185a7ae5a8d3a5611 in term 1.
I20260806 02:15:46.396354 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a30d673d56b4d2a9c39f7c8ca30a2d1" candidate_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f7277bf61e4409080ccc39f9540a80f"
I20260806 02:15:46.396409 3200 leader_election.cc:302] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fdec9444a874f5185a7ae5a8d3a5611, d502f01a8fd44cd08a55090c1958e1cf; no voters:
I20260806 02:15:46.396510 3116 raft_consensus.cc:3037] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:15:46.396577 3481 raft_consensus.cc:2781] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:15:46.396740 3481 raft_consensus.cc:686] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [term 1 LEADER]: Becoming Leader. State: Replica: 6fdec9444a874f5185a7ae5a8d3a5611, State: Running, Role: LEADER
I20260806 02:15:46.397091 3481 consensus_queue.cc:237] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:15:46.397724 3116 raft_consensus.cc:2454] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fdec9444a874f5185a7ae5a8d3a5611 in term 1.
I20260806 02:15:46.400287 2964 catalog_manager.cc:5495] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66). New cstate: current_term: 1 leader_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } health_report { overall_health: UNKNOWN } } }
W20260806 02:15:46.432126 3322 tablet.cc:2369] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:15:46.462550 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.459274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36720 (local address 127.18.70.66:42689)
0806 02:15:46.459420 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 02:15:46.459424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:15:46.459440 (+ 16us) server_negotiation.cc:407] Connection header received
0806 02:15:46.459516 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:15:46.459518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:15:46.459633 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:15:46.459727 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:15:46.459963 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.460494 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.461266 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:15:46.461470 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:15:46.461564 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:15:46.461826 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:15:46.462241 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:15:46.462344 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 02:15:46.462388 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 02:15:46.462436 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 02:15:46.468348 3412 raft_consensus.cc:1261] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf [term 1 FOLLOWER]: Refusing update from remote peer 6fdec9444a874f5185a7ae5a8d3a5611: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:15:46.468348 3116 raft_consensus.cc:1261] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Refusing update from remote peer 6fdec9444a874f5185a7ae5a8d3a5611: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:15:46.469132 3481 consensus_queue.cc:1035] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:15:46.469365 3485 consensus_queue.cc:1035] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:15:46.474777 3489 mvcc.cc:205] Tried to move back new op lower bound from 7315384510327308288 to 7315384510045544448. Current Snapshot: MvccSnapshot[applied={T|T < 7315384510327308288}]
I20260806 02:15:46.475178 3488 mvcc.cc:205] Tried to move back new op lower bound from 7315384510327308288 to 7315384510045544448. Current Snapshot: MvccSnapshot[applied={T|T < 7315384510327308288}]
I20260806 02:15:46.478796 3490 mvcc.cc:205] Tried to move back new op lower bound from 7315384510327308288 to 7315384510045544448. Current Snapshot: MvccSnapshot[applied={T|T < 7315384510327308288}]
I20260806 02:15:46.483533 2964 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:15:46.483742 2964 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:15:46.486204 2964 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Sending DeleteTablet for 3 replicas of tablet 8a30d673d56b4d2a9c39f7c8ca30a2d1
I20260806 02:15:46.486809 3247 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a30d673d56b4d2a9c39f7c8ca30a2d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:15:46 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:36704
I20260806 02:15:46.486967 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 3006
I20260806 02:15:46.489022 3498 tablet_replica.cc:330] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611: stopping tablet replica
I20260806 02:15:46.489384 3498 raft_consensus.cc:2229] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:15:46.489827 3498 raft_consensus.cc:2258] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:15:46.491402 3498 ts_tablet_manager.cc:1905] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:15:46.491370 3392 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a30d673d56b4d2a9c39f7c8ca30a2d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:15:46 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:48274
I20260806 02:15:46.492254 3499 tablet_replica.cc:330] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf: stopping tablet replica
I20260806 02:15:46.492539 3499 raft_consensus.cc:2229] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:15:46.492832 3499 raft_consensus.cc:2258] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:15:46.494277 3499 ts_tablet_manager.cc:1905] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:15:46.494654 3498 ts_tablet_manager.cc:1918] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:15:46.494798 3498 log.cc:1196] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/8a30d673d56b4d2a9c39f7c8ca30a2d1
I20260806 02:15:46.495317 3498 ts_tablet_manager.cc:1939] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 6fdec9444a874f5185a7ae5a8d3a5611: Deleting consensus metadata
I20260806 02:15:46.496537 2947 catalog_manager.cc:4841] TS 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689): tablet 8a30d673d56b4d2a9c39f7c8ca30a2d1 (table test-table [id=73733fb772e04311995f95392a25c972]) successfully deleted
I20260806 02:15:46.496937 3499 ts_tablet_manager.cc:1918] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:15:46.497028 3499 log.cc:1196] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/8a30d673d56b4d2a9c39f7c8ca30a2d1
I20260806 02:15:46.497335 3499 ts_tablet_manager.cc:1939] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P d502f01a8fd44cd08a55090c1958e1cf: Deleting consensus metadata
I20260806 02:15:46.498396 2950 catalog_manager.cc:4841] TS d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737): tablet 8a30d673d56b4d2a9c39f7c8ca30a2d1 (table test-table [id=73733fb772e04311995f95392a25c972]) successfully deleted
W20260806 02:15:46.505792 2949 connection.cc:535] client connection to 127.18.70.65:40809 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 02:15:46.505944 2949 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 02:15:46.506167 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 3174
I20260806 02:15:46.509258 3474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:15:46.508811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:40809 (local address 127.0.0.1:53432)
0806 02:15:46.509076 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 02:15:46.509166 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:40809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
W20260806 02:15:46.510108 2949 catalog_manager.cc:4584] TS 9f7277bf61e4409080ccc39f9540a80f (127.18.70.65:40809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a30d673d56b4d2a9c39f7c8ca30a2d1: Network error: Client connection negotiation failed: client connection to 127.18.70.65:40809: connect: Connection refused (error 111)
I20260806 02:15:46.515748 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 3324
I20260806 02:15:46.522354 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 2928
I20260806 02:16:06.530830 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:35757
--webserver_interface=127.18.70.126
--webserver_port=40031
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:35757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:06.644953 3501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:06.645234 3501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:06.645275 3501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:06.649279 3501 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 02:16:06.649334 3501 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 02:16:06.649350 3501 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 02:16:06.649363 3501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:06.649381 3501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:06.649398 3501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:06.649462 3501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:06.649498 3501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:06.654963 3501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:35757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:35757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=40031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:06.656073 3501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:06.657116 3501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:06.666167 3506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:06.666276 3501 server_base.cc:1034] running on GCE node
W20260806 02:16:06.666239 3507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:06.666463 3509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:06.667021 3501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:06.668226 3501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:06.669436 3501 hybrid_clock.cc:648] HybridClock initialized: now 1785982566669422 us; error 26 us; skew 500 ppm
Aug 06 02:16:06 dist-test-slave-6k22 krb5kdc[2326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982566, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:06.672295 3501 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:16:06.673347 3501 webserver.cc:460] Webserver started at http://127.18.70.126:40031/ using document root <none> and password file <none>
I20260806 02:16:06.673938 3501 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:06.674037 3501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:06.677846 3501 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260806 02:16:06.680660 3516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:06.682212 3501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 02:16:06.682360 3501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "6a71232f5f23d82bb6eae54d0e764ce2"
server_key_iv: "c07e18b46fac918e8b70c4924429ffd4"
server_key_version: "encryptionkey@0"
I20260806 02:16:06.682935 3501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:06.713593 3501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:06.717370 3501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:06.717618 3501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:06.725823 3501 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:35757
I20260806 02:16:06.726125 3575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:35757 every 8 connection(s)
I20260806 02:16:06.727063 3501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:06.728907 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 3501
I20260806 02:16:06.729414 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:40809
--local_ip_for_outbound_sockets=127.18.70.65
--tserver_master_addrs=127.18.70.126:35757
--webserver_port=35553
--webserver_interface=127.18.70.65
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:16:06.732646 3576 sys_catalog.cc:263] Verifying existing consensus state
I20260806 02:16:06.734290 3576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Bootstrap starting.
I20260806 02:16:06.752794 3576 log.cc:824] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:06.764219 3576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:16:06.764623 3576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Bootstrap complete.
I20260806 02:16:06.768862 3576 raft_consensus.cc:348] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } }
I20260806 02:16:06.769240 3576 raft_consensus.cc:729] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e370ecdf144c4b5facc77b7ecbc9c2cc, State: Initialized, Role: FOLLOWER
I20260806 02:16:06.769791 3576 consensus_queue.cc:260] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } }
I20260806 02:16:06.769909 3576 raft_consensus.cc:388] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:06.770002 3576 raft_consensus.cc:482] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:06.770126 3576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 2 FOLLOWER]: Advancing to term 3
I20260806 02:16:06.772426 3576 raft_consensus.cc:504] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } }
I20260806 02:16:06.772831 3576 leader_election.cc:302] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e370ecdf144c4b5facc77b7ecbc9c2cc; no voters:
I20260806 02:16:06.773185 3576 leader_election.cc:288] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 02:16:06.773378 3581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 3 FOLLOWER]: Leader election won for term 3
I20260806 02:16:06.773707 3581 raft_consensus.cc:686] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [term 3 LEADER]: Becoming Leader. State: Replica: e370ecdf144c4b5facc77b7ecbc9c2cc, State: Running, Role: LEADER
I20260806 02:16:06.774035 3581 consensus_queue.cc:237] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } }
I20260806 02:16:06.774601 3576 sys_catalog.cc:564] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:06.775141 3583 sys_catalog.cc:455] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader e370ecdf144c4b5facc77b7ecbc9c2cc. Latest consensus state: current_term: 3 leader_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } } }
I20260806 02:16:06.775276 3583 sys_catalog.cc:458] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:06.775507 3582 sys_catalog.cc:455] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e370ecdf144c4b5facc77b7ecbc9c2cc" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35757 } } }
I20260806 02:16:06.775657 3582 sys_catalog.cc:458] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:06.775979 3585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:06.781742 3585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:06.782476 3585 catalog_manager.cc:1259] Loaded cluster ID: c8298e54946c484580cd31c97823b030
I20260806 02:16:06.782545 3585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:06.785392 3585 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:06.786144 3585 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Loaded TSK: 0
I20260806 02:16:06.786230 3585 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e370ecdf144c4b5facc77b7ecbc9c2cc: Loaded TSK: 1
F20260806 02:16:06.787096 3585 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb519a959ad google::LogMessage::Fail() at ??:0
@ 0x7fb519a9888a google::LogMessage::SendToLog() at ??:0
@ 0x7fb519a954de google::LogMessage::Flush() at ??:0
@ 0x7fb519a96a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb51c75918d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb51c751f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb5243fa7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fb5243d499c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb5243d5178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fb524416df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fb52443582b std::function<>::operator()() at ??:0
@ 0x7fb5243d4b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fb5243d5f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb5243d2579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fb524416640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fb51bce73d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb51bce7c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fb51bce93f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fb51bcd8512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb51d31c6db start_thread at ??:0
@ 0x7fb51859361f clone at ??:0
W20260806 02:16:06.873198 3579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:06.873397 3579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:06.873435 3579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:06.877183 3579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:06.877224 3579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:06.877279 3579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:16:06.882701 3579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:40809
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=35553
--tserver_master_addrs=127.18.70.126:35757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:06.883795 3579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:06.884752 3579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:06.894302 3627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:06.894445 3625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:06.894325 3629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:06.895377 3579 server_base.cc:1034] running on GCE node
I20260806 02:16:06.895812 3579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:06.896461 3579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:06.897619 3579 hybrid_clock.cc:648] HybridClock initialized: now 1785982566897588 us; error 55 us; skew 500 ppm
Aug 06 02:16:06 dist-test-slave-6k22 krb5kdc[2326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982566, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:06.901014 3579 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:16:06.901947 3579 webserver.cc:460] Webserver started at http://127.18.70.65:35553/ using document root <none> and password file <none>
I20260806 02:16:06.902547 3579 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:06.902649 3579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:06.906692 3579 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 02:16:06.909067 3637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:06.910434 3579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 02:16:06.910535 3579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "9f7277bf61e4409080ccc39f9540a80f"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "5213d070314541d6d147277d1e58ed77"
server_key_iv: "f18ce112d48718eab89c9c92f329ca99"
server_key_version: "encryptionkey@0"
I20260806 02:16:06.911108 3579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:06.924997 3579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:06.928436 3579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:06.928653 3579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:06.929278 3579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:06.930384 3644 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 02:16:06.935766 3579 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 02:16:06.935847 3579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 02:16:06.935933 3579 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 02:16:06.937624 3579 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 02:16:06.937702 3579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 02:16:06.938006 3644 tablet_bootstrap.cc:492] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f: Bootstrap starting.
I20260806 02:16:06.948799 3579 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:40809
I20260806 02:16:06.948855 3756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:40809 every 8 connection(s)
I20260806 02:16:06.949857 3579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:16:06.959888 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 3579
I20260806 02:16:06.960346 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:42689
--local_ip_for_outbound_sockets=127.18.70.66
--tserver_master_addrs=127.18.70.126:35757
--webserver_port=34571
--webserver_interface=127.18.70.66
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:16:06.960927 3644 log.cc:824] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:06.970463 3644 tablet_bootstrap.cc:492] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 02:16:06.970942 3644 tablet_bootstrap.cc:492] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f: Bootstrap complete.
I20260806 02:16:06.972622 3644 ts_tablet_manager.cc:1397] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f: Time spent bootstrapping tablet: real 0.035s user 0.026s sys 0.007s
I20260806 02:16:06.975935 3644 raft_consensus.cc:348] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:16:06.976516 3644 raft_consensus.cc:729] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f7277bf61e4409080ccc39f9540a80f, State: Initialized, Role: FOLLOWER
I20260806 02:16:06.977053 3644 consensus_queue.cc:260] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:16:06.978143 3644 ts_tablet_manager.cc:1428] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260806 02:16:07.053728 3760 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:16:06.951847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:36447)
0806 02:16:06.952380 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:06.952437 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:06.953253 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:07.052703 (+ 99450us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":156,"threads_started":1}
W20260806 02:16:07.054376 3758 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:35757 (0 consecutive failures): Network error: Failed to ping master at 127.18.70.126:35757: Client connection negotiation failed: client connection to 127.18.70.126:35757: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 02:16:07.055204 3760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:07.054895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:58033)
0806 02:16:07.055054 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:07.055136 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 02:16:07.055975 3760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:07.055743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:38675)
0806 02:16:07.055867 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:07.055917 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 02:16:07.056190 3758 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 02:16:07.083526 3763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:07.083841 3763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:07.083887 3763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:07.088052 3763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:07.088099 3763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:07.088217 3763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:16:07.093606 3763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:42689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=34571
--tserver_master_addrs=127.18.70.126:35757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:07.094594 3763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:07.095661 3763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:07.105420 3769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:07.105420 3770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:07.105571 3773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:07.105445 3763 server_base.cc:1034] running on GCE node
I20260806 02:16:07.106024 3763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:07.106632 3763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:07.107802 3763 hybrid_clock.cc:648] HybridClock initialized: now 1785982567107788 us; error 24 us; skew 500 ppm
Aug 06 02:16:07 dist-test-slave-6k22 krb5kdc[2326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982567, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:07.110436 3763 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:16:07.111284 3763 webserver.cc:460] Webserver started at http://127.18.70.66:34571/ using document root <none> and password file <none>
I20260806 02:16:07.111899 3763 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:07.111976 3763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:07.115912 3763 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:16:07.117890 3780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:07.119223 3763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:16:07.119364 3763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "6fdec9444a874f5185a7ae5a8d3a5611"
format_stamp: "Formatted at 2026-08-06 02:15:24 on dist-test-slave-6k22"
server_key: "e85c266fc46503137064412a508f4343"
server_key_iv: "27a9d48d6c579fd9323206893489e047"
server_key_version: "encryptionkey@0"
I20260806 02:16:07.119984 3763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:07.133723 3763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:07.136534 3763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:07.136742 3763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:07.137320 3763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:07.138144 3763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:07.138221 3763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:07.138285 3763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:07.138339 3763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:07.147531 3763 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:42689
I20260806 02:16:07.147621 3900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:42689 every 8 connection(s)
I20260806 02:16:07.148582 3763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:16:07.152364 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 3763
I20260806 02:16:07.152961 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:35737
--local_ip_for_outbound_sockets=127.18.70.67
--tserver_master_addrs=127.18.70.126:35757
--webserver_port=38177
--webserver_interface=127.18.70.67
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:16:07.153200 3904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:07.151012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:49325)
0806 02:16:07.151496 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:07.151666 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":125,"threads_started":1}
W20260806 02:16:07.153697 3902 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:35757 (0 consecutive failures): Network error: Failed to ping master at 127.18.70.126:35757: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
I20260806 02:16:07.154383 3904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:07.154148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:54709)
0806 02:16:07.154271 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:07.154323 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 02:16:07.155196 3904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:07.154957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:48051)
0806 02:16:07.155078 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:07.155142 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 02:16:07.155480 3902 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 02:16:07.267719 3905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:07.267953 3905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:07.267994 3905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:07.272010 3905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:07.272065 3905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:07.272148 3905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:16:07.277359 3905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:35737
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=38177
--tserver_master_addrs=127.18.70.126:35757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:07.278404 3905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:07.279487 3905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:07.289520 3912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:07.289525 3914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:07.289564 3910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:07.289795 3905 server_base.cc:1034] running on GCE node
I20260806 02:16:07.290343 3905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:07.291182 3905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:07.292382 3905 hybrid_clock.cc:648] HybridClock initialized: now 1785982567292348 us; error 43 us; skew 500 ppm
Aug 06 02:16:07 dist-test-slave-6k22 krb5kdc[2326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982567, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:07.295208 3905 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:16:07.296135 3905 webserver.cc:460] Webserver started at http://127.18.70.67:38177/ using document root <none> and password file <none>
I20260806 02:16:07.296718 3905 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:07.296787 3905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:07.300689 3905 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:16:07.302711 3921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:07.304441 3905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:16:07.304587 3905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "d502f01a8fd44cd08a55090c1958e1cf"
format_stamp: "Formatted at 2026-08-06 02:15:25 on dist-test-slave-6k22"
server_key: "0483c2fd2ad70904616567362629dbf0"
server_key_iv: "e79609d1f2111060f6186231de428dac"
server_key_version: "encryptionkey@0"
I20260806 02:16:07.305114 3905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:07.334995 3905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:07.338477 3905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:07.338723 3905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:07.339411 3905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:07.340391 3905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:07.340485 3905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:07.340559 3905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:07.340610 3905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:07.350256 3905 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:35737
I20260806 02:16:07.350281 4050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:35737 every 8 connection(s)
I20260806 02:16:07.351372 3905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:16:07.355479 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:07.353744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:39495)
0806 02:16:07.354246 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:07.354352 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":120,"threads_started":1}
W20260806 02:16:07.355984 4051 heartbeater.cc:644] Failed to heartbeat to 127.18.70.126:35757 (0 consecutive failures): Network error: Failed to ping master at 127.18.70.126:35757: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
I20260806 02:16:07.356557 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:07.356348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:47859)
0806 02:16:07.356450 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:07.356507 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 02:16:07.357218 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:07.356994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:37549)
0806 02:16:07.357114 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:07.357163 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 02:16:07.357486 4051 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 02:16:07.360859 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 3905
I20260806 02:16:08.057215 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:08.056796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:54061)
0806 02:16:08.057060 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:08.057111 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":76,"threads_started":1}
I20260806 02:16:08.156491 4056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:08.156073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:53897)
0806 02:16:08.156322 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:08.156382 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":84,"threads_started":1}
I20260806 02:16:08.324854 4057 raft_consensus.cc:482] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:16:08.325049 4057 raft_consensus.cc:504] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:16:08.326184 4057 leader_election.cc:288] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689), d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737)
Aug 06 02:16:08 dist-test-slave-6k22 krb5kdc[2326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982566, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.67@KRBTEST.COM
Aug 06 02:16:08 dist-test-slave-6k22 krb5kdc[2326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982566, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.66@KRBTEST.COM
I20260806 02:16:08.341423 4055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:08.326331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:42689 (local address 127.18.70.65:59823)
0806 02:16:08.326463 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:08.326479 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:08.326584 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:08.328819 (+ 2235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:08.328824 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:08.329264 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:08.329818 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:08.329834 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:08.331132 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:08.331136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:08.332187 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:08.332197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:08.333291 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:08.333294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:08.333369 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:08.334024 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:16:08.334036 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:08.335997 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:08.338294 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:08.338301 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:08.338317 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:08.338845 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:08.339280 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:08.339283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:08.339286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:08.339517 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:08.340308 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:08.340315 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:08.340658 (+ 343us) client_negotiation.cc:769] Sending connection context
0806 02:16:08.340871 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 02:16:08.341269 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":258,"spinlock_wait_cycles":57856}
I20260806 02:16:08.341424 4058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:08.326439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:35737 (local address 127.18.70.65:51459)
0806 02:16:08.326752 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:08.326777 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:08.326881 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:08.329265 (+ 2384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:08.329267 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:08.329439 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:08.329818 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:08.329833 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:08.331128 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:08.331131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:08.332231 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:08.332243 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:08.333049 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:08.333053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:08.333220 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:08.333891 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:16:08.333917 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:08.335603 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:08.338294 (+ 2691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:08.338301 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:08.338317 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:08.338845 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:08.339280 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:08.339283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:08.339285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:08.339517 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:08.340308 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:08.340315 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:08.340658 (+ 343us) client_negotiation.cc:769] Sending connection context
0806 02:16:08.340871 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 02:16:08.341222 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":264,"spinlock_wait_cycles":11648,"thread_start_us":77,"threads_started":1}
I20260806 02:16:08.341509 4060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:08.326702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:51459 (local address 127.18.70.67:35737)
0806 02:16:08.327008 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:08.327019 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:08.327048 (+ 29us) server_negotiation.cc:407] Connection header received
0806 02:16:08.327867 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:08.327877 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:08.328369 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:08.328726 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:08.330006 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:08.330981 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:08.332345 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:08.332916 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:08.333102 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:08.335893 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:08.335898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:08.335910 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:08.335955 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:08.338124 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:08.338974 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:08.338981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:08.338987 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:08.339152 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:08.339636 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:08.339640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:08.339641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:08.340154 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:08.340295 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:08.341037 (+ 742us) server_negotiation.cc:299] Negotiation successful
0806 02:16:08.341336 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":71,"threads_started":1}
I20260806 02:16:08.342057 4059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:08.326542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:59823 (local address 127.18.70.66:42689)
0806 02:16:08.326827 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:08.326871 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:08.326918 (+ 47us) server_negotiation.cc:407] Connection header received
0806 02:16:08.327875 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:08.327882 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:08.328369 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:08.328727 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:08.330002 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:08.330959 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:08.332609 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:08.333159 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:08.333337 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:08.336200 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:08.336208 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:08.336224 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:08.336263 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:08.338137 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:08.338974 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:08.338981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:08.338987 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:08.339152 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:08.339640 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:08.339643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:08.339645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:08.340153 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:08.340295 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:08.341036 (+ 741us) server_negotiation.cc:299] Negotiation successful
0806 02:16:08.341331 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":112,"threads_started":1}
W20260806 02:16:08.344313 3641 leader_election.cc:341] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737): Not found: Tablet not found: 8a30d673d56b4d2a9c39f7c8ca30a2d1
W20260806 02:16:08.344491 3638 leader_election.cc:341] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689): Not found: Tablet not found: 8a30d673d56b4d2a9c39f7c8ca30a2d1
I20260806 02:16:08.344619 3638 leader_election.cc:302] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f7277bf61e4409080ccc39f9540a80f; no voters: 6fdec9444a874f5185a7ae5a8d3a5611, d502f01a8fd44cd08a55090c1958e1cf
I20260806 02:16:08.344894 4057 raft_consensus.cc:2726] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:16:08.358331 4061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:08.357973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:39177)
0806 02:16:08.358192 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:08.358248 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":66,"threads_started":1}
I20260806 02:16:09.058570 4062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:09.058086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:49691)
0806 02:16:09.058388 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:09.058455 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":83,"threads_started":1}
I20260806 02:16:09.157793 4063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:09.157327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:37121)
0806 02:16:09.157637 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:09.157699 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":73,"threads_started":1}
I20260806 02:16:09.359443 4064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:09.359021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:33145)
0806 02:16:09.359278 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:09.359335 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":72,"threads_started":1}
I20260806 02:16:10.060247 4065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:10.059774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:46583)
0806 02:16:10.060078 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:10.060142 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":97,"threads_started":1}
I20260806 02:16:10.158986 4066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:10.158543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:56517)
0806 02:16:10.158839 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:10.158894 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":109,"threads_started":1}
I20260806 02:16:10.328104 4067 raft_consensus.cc:482] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:16:10.328238 4067 raft_consensus.cc:504] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:16:10.328724 4067 leader_election.cc:288] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689), d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737)
W20260806 02:16:10.329536 3641 leader_election.cc:341] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737): Not found: Tablet not found: 8a30d673d56b4d2a9c39f7c8ca30a2d1
W20260806 02:16:10.329661 3638 leader_election.cc:341] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689): Not found: Tablet not found: 8a30d673d56b4d2a9c39f7c8ca30a2d1
I20260806 02:16:10.329762 3638 leader_election.cc:302] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f7277bf61e4409080ccc39f9540a80f; no voters: 6fdec9444a874f5185a7ae5a8d3a5611, d502f01a8fd44cd08a55090c1958e1cf
I20260806 02:16:10.329958 4067 raft_consensus.cc:2726] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:16:10.360550 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:10.360166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:47973)
0806 02:16:10.360422 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:10.360471 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":82,"threads_started":1}
I20260806 02:16:11.061496 4069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:11.061077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:52939)
0806 02:16:11.061339 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:11.061396 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":85,"threads_started":1}
I20260806 02:16:11.160205 4070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:11.159804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:56921)
0806 02:16:11.160057 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:11.160109 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":82,"threads_started":1}
I20260806 02:16:11.361687 4071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:11.361286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:37557)
0806 02:16:11.361540 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:11.361594 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":75,"threads_started":1}
I20260806 02:16:12.063062 4072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:12.062493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:33987)
0806 02:16:12.062874 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:12.062939 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":96,"threads_started":1}
I20260806 02:16:12.161435 4073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:12.161003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:46599)
0806 02:16:12.161273 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:12.161332 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":93,"threads_started":1}
I20260806 02:16:12.362967 4074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:12.362522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:39285)
0806 02:16:12.362807 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:12.362871 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":77,"threads_started":1}
I20260806 02:16:12.942572 4075 raft_consensus.cc:482] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:16:12.942816 4075 raft_consensus.cc:504] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:16:12.943250 4075 leader_election.cc:288] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689), d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737)
W20260806 02:16:12.944193 3638 leader_election.cc:341] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689): Not found: Tablet not found: 8a30d673d56b4d2a9c39f7c8ca30a2d1
W20260806 02:16:12.944306 3641 leader_election.cc:341] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737): Not found: Tablet not found: 8a30d673d56b4d2a9c39f7c8ca30a2d1
I20260806 02:16:12.944367 3641 leader_election.cc:302] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f7277bf61e4409080ccc39f9540a80f; no voters: 6fdec9444a874f5185a7ae5a8d3a5611, d502f01a8fd44cd08a55090c1958e1cf
I20260806 02:16:12.944530 4075 raft_consensus.cc:2726] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:16:13.064524 4076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:13.063980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:38105)
0806 02:16:13.064319 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:13.064379 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":106,"threads_started":1}
I20260806 02:16:13.163074 4077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:13.162660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:48397)
0806 02:16:13.162916 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:13.162966 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":90,"threads_started":1}
I20260806 02:16:13.364131 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:13.363730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:56361)
0806 02:16:13.363989 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:13.364043 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":75,"threads_started":1}
I20260806 02:16:14.065747 4079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:14.065317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:45363)
0806 02:16:14.065589 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:14.065646 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":89,"threads_started":1}
I20260806 02:16:14.164271 4080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:14.163827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:41465)
0806 02:16:14.164121 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:14.164175 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":73,"threads_started":1}
I20260806 02:16:14.365398 4082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:14.364918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:51749)
0806 02:16:14.365226 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:14.365301 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":74,"threads_started":1}
I20260806 02:16:15.066931 4083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:15.066503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:49401)
0806 02:16:15.066770 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:15.066829 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":92,"threads_started":1}
I20260806 02:16:15.165446 4084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:15.165019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:47747)
0806 02:16:15.165303 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:15.165354 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":113,"threads_started":1}
I20260806 02:16:15.366834 4085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:15.366386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:33081)
0806 02:16:15.366644 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:15.366721 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":95,"threads_started":1}
I20260806 02:16:16.068257 4086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:16.067818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:53833)
0806 02:16:16.068099 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:16.068154 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":97,"threads_started":1}
I20260806 02:16:16.166627 4087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:16.166209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:57131)
0806 02:16:16.166476 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:16.166530 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":104,"threads_started":1}
I20260806 02:16:16.368065 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:16.367645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:35665)
0806 02:16:16.367905 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:16.367961 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":87,"threads_started":1}
I20260806 02:16:17.069511 4089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:17.069100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:57947)
0806 02:16:17.069361 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:17.069413 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":81,"threads_started":1}
I20260806 02:16:17.167974 4090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:17.167513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:51569)
0806 02:16:17.167831 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:17.167882 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":160,"threads_started":1}
I20260806 02:16:17.350931 4091 raft_consensus.cc:482] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:16:17.351064 4091 raft_consensus.cc:504] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fdec9444a874f5185a7ae5a8d3a5611" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 42689 } } peers { permanent_uuid: "d502f01a8fd44cd08a55090c1958e1cf" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 35737 } } peers { permanent_uuid: "9f7277bf61e4409080ccc39f9540a80f" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 40809 } }
I20260806 02:16:17.351544 4091 leader_election.cc:288] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689), d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737)
W20260806 02:16:17.352305 3641 leader_election.cc:341] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d502f01a8fd44cd08a55090c1958e1cf (127.18.70.67:35737): Not found: Tablet not found: 8a30d673d56b4d2a9c39f7c8ca30a2d1
W20260806 02:16:17.352406 3638 leader_election.cc:341] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6fdec9444a874f5185a7ae5a8d3a5611 (127.18.70.66:42689): Not found: Tablet not found: 8a30d673d56b4d2a9c39f7c8ca30a2d1
I20260806 02:16:17.352523 3638 leader_election.cc:302] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f7277bf61e4409080ccc39f9540a80f; no voters: 6fdec9444a874f5185a7ae5a8d3a5611, d502f01a8fd44cd08a55090c1958e1cf
I20260806 02:16:17.352685 4091 raft_consensus.cc:2726] T 8a30d673d56b4d2a9c39f7c8ca30a2d1 P 9f7277bf61e4409080ccc39f9540a80f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 02:16:17.369143 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:17.368757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:54203)
0806 02:16:17.369001 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:17.369055 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
I20260806 02:16:18.070729 4093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:18.070303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:38939)
0806 02:16:18.070575 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:18.070630 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":96,"threads_started":1}
I20260806 02:16:18.169095 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:18.168655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:51613)
0806 02:16:18.168946 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:18.169001 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":82,"threads_started":1}
I20260806 02:16:18.370639 4095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:18.370030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:42219)
0806 02:16:18.370438 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:18.370510 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":88,"threads_started":1}
I20260806 02:16:19.072141 4096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:19.071693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:45099)
0806 02:16:19.071987 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:19.072046 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":84,"threads_started":1}
I20260806 02:16:19.170267 4097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:19.169839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:36711)
0806 02:16:19.170118 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:19.170167 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":127,"threads_started":1}
I20260806 02:16:19.371845 4099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:19.371426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:39363)
0806 02:16:19.371700 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:19.371752 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":71,"threads_started":1}
I20260806 02:16:20.073342 4101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:20.072910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:55451)
0806 02:16:20.073180 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:20.073233 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":112,"threads_started":1}
I20260806 02:16:20.171449 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:20.171067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:39653)
0806 02:16:20.171303 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:20.171353 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":72,"threads_started":1}
I20260806 02:16:20.373116 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:20.372685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:53683)
0806 02:16:20.372968 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:20.373020 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":93,"threads_started":1}
I20260806 02:16:21.074482 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:21.074096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:45543)
0806 02:16:21.074334 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:21.074387 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260806 02:16:21.172572 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:21.172181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:47787)
0806 02:16:21.172402 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:21.172449 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":69,"threads_started":1}
I20260806 02:16:21.374291 4107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:21.373897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:47285)
0806 02:16:21.374147 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:21.374201 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":92,"threads_started":1}
I20260806 02:16:22.075922 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:22.075393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.65:46863)
0806 02:16:22.075774 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:22.075826 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":71,"threads_started":1}
I20260806 02:16:22.173955 4109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:22.173468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.66:35671)
0806 02:16:22.173747 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:22.173807 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":71,"threads_started":1}
I20260806 02:16:22.371592 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 3579
I20260806 02:16:22.375626 4110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:22.375200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35757 (local address 127.18.70.67:42831)
0806 02:16:22.375438 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:22.375489 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.126:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":88,"threads_started":1}
I20260806 02:16:22.380962 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 3763
I20260806 02:16:22.387377 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 3905
2026-08-06T02:16:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59510 ms)
[----------] 30 tests from SecurityITest (242598 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:22 dist-test-slave-6k22 krb5kdc[4114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:22 dist-test-slave-6k22 krb5kdc[4114](info): set up 2 sockets
Aug 06 02:16:22 dist-test-slave-6k22 krb5kdc[4114](info): commencing operation
krb5kdc: starting...
W20260806 02:16:23.452960 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:23 dist-test-slave-6k22 krb5kdc[4114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:16:23.577062 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:42301
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:37415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:42301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:23.694569 4127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:23.694869 4127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:23.694965 4127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:23.699340 4127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:23.699437 4127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:23.699478 4127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:23.699514 4127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:23.699608 4127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:23.705260 4127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:42301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:42301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:23.706336 4127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:23.707367 4127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:23.715963 4136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:23.715968 4133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:23.716141 4127 server_base.cc:1034] running on GCE node
W20260806 02:16:23.716027 4134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:23.716904 4127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:23.718029 4127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:23.719235 4127 hybrid_clock.cc:648] HybridClock initialized: now 1785982583719224 us; error 30 us; skew 500 ppm
Aug 06 02:16:23 dist-test-slave-6k22 krb5kdc[4114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982583, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:23.722048 4127 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:16:23.723001 4127 webserver.cc:460] Webserver started at http://127.18.70.126:39491/ using document root <none> and password file <none>
I20260806 02:16:23.723631 4127 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:23.723757 4127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:23.724002 4127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:23.726135 4127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "4c91dc93342e420bbc40d90d3838d4a2"
format_stamp: "Formatted at 2026-08-06 02:16:23 on dist-test-slave-6k22"
server_key: "67bfc93be5aab12d5893f88d7f2cb802"
server_key_iv: "8a407378e51925c66bd86d8b3e0dfbb2"
server_key_version: "encryptionkey@0"
I20260806 02:16:23.726722 4127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "4c91dc93342e420bbc40d90d3838d4a2"
format_stamp: "Formatted at 2026-08-06 02:16:23 on dist-test-slave-6k22"
server_key: "67bfc93be5aab12d5893f88d7f2cb802"
server_key_iv: "8a407378e51925c66bd86d8b3e0dfbb2"
server_key_version: "encryptionkey@0"
I20260806 02:16:23.730608 4127 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260806 02:16:23.733199 4144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:23.734540 4127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:16:23.734725 4127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "4c91dc93342e420bbc40d90d3838d4a2"
format_stamp: "Formatted at 2026-08-06 02:16:23 on dist-test-slave-6k22"
server_key: "67bfc93be5aab12d5893f88d7f2cb802"
server_key_iv: "8a407378e51925c66bd86d8b3e0dfbb2"
server_key_version: "encryptionkey@0"
I20260806 02:16:23.734855 4127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:23.784869 4127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:23.787997 4127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:23.788223 4127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:23.795835 4127 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:42301
I20260806 02:16:23.795877 4198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:42301 every 8 connection(s)
I20260806 02:16:23.796857 4127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:23.799670 4199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:23.805058 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 4127
I20260806 02:16:23.805167 4199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2: Bootstrap starting.
I20260806 02:16:23.805205 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:23.805557 18713 external_mini_cluster.cc:1428] Setting key 4d95e311cf809b0772b9d2a755069228
I20260806 02:16:23.807837 4199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:23.808610 4199 log.cc:824] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:23.810461 4199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2: No bootstrap required, opened a new log
I20260806 02:16:23.814014 4199 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c91dc93342e420bbc40d90d3838d4a2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42301 } }
I20260806 02:16:23.814333 4199 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:23.814405 4199 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c91dc93342e420bbc40d90d3838d4a2, State: Initialized, Role: FOLLOWER
Aug 06 02:16:23 dist-test-slave-6k22 krb5kdc[4114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:16:23.814926 4199 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c91dc93342e420bbc40d90d3838d4a2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42301 } }
I20260806 02:16:23.815114 4199 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:23.815186 4199 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:23.815292 4199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:23.816474 4199 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c91dc93342e420bbc40d90d3838d4a2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42301 } }
I20260806 02:16:23.816885 4199 leader_election.cc:302] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c91dc93342e420bbc40d90d3838d4a2; no voters:
I20260806 02:16:23.817224 4199 leader_election.cc:288] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:23.817298 4204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:23.817523 4204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [term 1 LEADER]: Becoming Leader. State: Replica: 4c91dc93342e420bbc40d90d3838d4a2, State: Running, Role: LEADER
I20260806 02:16:23.817865 4204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c91dc93342e420bbc40d90d3838d4a2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42301 } }
I20260806 02:16:23.818496 4199 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:23.819495 4205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c91dc93342e420bbc40d90d3838d4a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c91dc93342e420bbc40d90d3838d4a2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42301 } } }
I20260806 02:16:23.819648 4205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:23.819878 4205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c91dc93342e420bbc40d90d3838d4a2. Latest consensus state: current_term: 1 leader_uuid: "4c91dc93342e420bbc40d90d3838d4a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c91dc93342e420bbc40d90d3838d4a2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 42301 } } }
I20260806 02:16:23.819952 4205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:23.820518 4214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:23.821691 4202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:23.806937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35642 (local address 127.18.70.126:42301)
0806 02:16:23.807378 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:23.807393 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:23.807429 (+ 36us) server_negotiation.cc:407] Connection header received
0806 02:16:23.808166 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:23.808185 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:23.808567 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:23.808958 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:23.809858 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:23.811003 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:23.811947 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:23.812429 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:23.812607 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:23.815394 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:23.815402 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:23.815419 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:23.815454 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:23.817500 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:23.818096 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:23.818101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:23.818105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:23.818219 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:23.818528 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:23.818531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:23.818533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:23.819101 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:23.820086 (+ 985us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:23.820242 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 02:16:23.820545 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":126,"threads_started":1}
I20260806 02:16:23.824627 4214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:23.830448 4214 catalog_manager.cc:1347] Generated new cluster ID: ba017fc0f73e4e54902e766e74136e1d
I20260806 02:16:23.830513 4214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:23.856195 4214 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:23.857132 4214 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:23.872728 4214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2: Generated new TSK 0
I20260806 02:16:23.873443 4214 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:16:23.912348 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42301
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:37415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:24.058259 4227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:24.058633 4227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:24.058753 4227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:24.063105 4227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:24.063155 4227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:24.063244 4227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:16:24.068384 4227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:24.069499 4227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:24.070513 4227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:24.079909 4233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:24.079893 4235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:24.079897 4237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:24.080197 4227 server_base.cc:1034] running on GCE node
I20260806 02:16:24.080601 4227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:24.081233 4227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:24.082412 4227 hybrid_clock.cc:648] HybridClock initialized: now 1785982584082393 us; error 30 us; skew 500 ppm
Aug 06 02:16:24 dist-test-slave-6k22 krb5kdc[4114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982584, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:24.085152 4227 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:16:24.085983 4227 webserver.cc:460] Webserver started at http://127.18.70.65:33955/ using document root <none> and password file <none>
I20260806 02:16:24.086494 4227 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:24.086548 4227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:24.086716 4227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:24.088672 4227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "d7508c100b7d436596200d7aef62ee76"
format_stamp: "Formatted at 2026-08-06 02:16:24 on dist-test-slave-6k22"
server_key: "bcfd693da0f1705a97111fffb7b7186e"
server_key_iv: "9dc7be3992529b517d61e5061ae43d8a"
server_key_version: "encryptionkey@0"
I20260806 02:16:24.089229 4227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "d7508c100b7d436596200d7aef62ee76"
format_stamp: "Formatted at 2026-08-06 02:16:24 on dist-test-slave-6k22"
server_key: "bcfd693da0f1705a97111fffb7b7186e"
server_key_iv: "9dc7be3992529b517d61e5061ae43d8a"
server_key_version: "encryptionkey@0"
I20260806 02:16:24.092912 4227 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:16:24.095230 4245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:24.096458 4227 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:16:24.096575 4227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "d7508c100b7d436596200d7aef62ee76"
format_stamp: "Formatted at 2026-08-06 02:16:24 on dist-test-slave-6k22"
server_key: "bcfd693da0f1705a97111fffb7b7186e"
server_key_iv: "9dc7be3992529b517d61e5061ae43d8a"
server_key_version: "encryptionkey@0"
I20260806 02:16:24.096670 4227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:24.116813 4227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:24.120198 4227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:24.120417 4227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:24.121037 4227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:24.121834 4227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:24.121879 4227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:24.121942 4227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:24.121973 4227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:24.130796 4227 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:46387
I20260806 02:16:24.130839 4368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:46387 every 8 connection(s)
I20260806 02:16:24.131742 4227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:16:24.139866 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 4227
I20260806 02:16:24.140002 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:16:24.140333 18713 external_mini_cluster.cc:1428] Setting key 96d743178adb5a70bd3b35d59d9d3244
Aug 06 02:16:24 dist-test-slave-6k22 krb5kdc[4114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982584, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:16:24.145814 4202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.133411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:47097 (local address 127.18.70.126:42301)
0806 02:16:24.133541 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:24.133545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:24.134375 (+ 830us) server_negotiation.cc:407] Connection header received
0806 02:16:24.135776 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:24.135779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:24.135854 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:24.135955 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:24.137062 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.137579 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.138726 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.139030 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.139143 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.141822 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:24.141826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:24.141829 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:24.141856 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:24.143254 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:24.144095 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:24.144100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:24.144103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:24.144234 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:24.144591 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:24.144597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:24.144599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:24.144864 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:24.144969 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:24.145446 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 02:16:24.145591 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:16:24.146404 4371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.133765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42301 (local address 127.18.70.65:47097)
0806 02:16:24.134238 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:24.134279 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:24.135504 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:24.136126 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:24.136132 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:24.136472 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:24.136912 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.136925 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.137727 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.137730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.138597 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.138606 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.139182 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.139187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.139353 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.139913 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:16:24.139937 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:24.141659 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:24.143394 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:24.143403 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:24.143429 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:24.143850 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:24.144343 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:24.144345 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:24.144347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:24.144489 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:24.144967 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:24.144972 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:24.145187 (+ 215us) client_negotiation.cc:769] Sending connection context
0806 02:16:24.145345 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 02:16:24.145580 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":136,"threads_started":1}
I20260806 02:16:24.147667 4369 heartbeater.cc:347] Connected to a master server at 127.18.70.126:42301
I20260806 02:16:24.147927 4369 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:24.148521 4369 heartbeater.cc:511] Master 127.18.70.126:42301 requested a full tablet report, sending...
I20260806 02:16:24.150099 4161 ts_manager.cc:194] Registered new tserver with Master: d7508c100b7d436596200d7aef62ee76 (127.18.70.65:46387)
I20260806 02:16:24.151189 4161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:47097
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:16:24.169591 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42301
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:37415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:24.280674 4374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:24.280913 4374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:24.281198 4374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:24.285460 4374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:24.285521 4374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:24.285640 4374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:16:24.291100 4374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:24.292169 4374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:24.293183 4374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:24.302716 4384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:24.302685 4381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:24.302690 4382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:24.302950 4374 server_base.cc:1034] running on GCE node
I20260806 02:16:24.303429 4374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:24.304136 4374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:24.305290 4374 hybrid_clock.cc:648] HybridClock initialized: now 1785982584305277 us; error 23 us; skew 500 ppm
Aug 06 02:16:24 dist-test-slave-6k22 krb5kdc[4114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982584, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:24.307988 4374 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:16:24.308898 4374 webserver.cc:460] Webserver started at http://127.18.70.66:36409/ using document root <none> and password file <none>
I20260806 02:16:24.309445 4374 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:24.309504 4374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:24.309710 4374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:24.311746 4374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "461b4acf249b49ce9fefa8ed4bf60e57"
format_stamp: "Formatted at 2026-08-06 02:16:24 on dist-test-slave-6k22"
server_key: "ba7587db325023b3922e62ef64f565f7"
server_key_iv: "de3e2a418bf5663717998c0b9e54bf2e"
server_key_version: "encryptionkey@0"
I20260806 02:16:24.312305 4374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "461b4acf249b49ce9fefa8ed4bf60e57"
format_stamp: "Formatted at 2026-08-06 02:16:24 on dist-test-slave-6k22"
server_key: "ba7587db325023b3922e62ef64f565f7"
server_key_iv: "de3e2a418bf5663717998c0b9e54bf2e"
server_key_version: "encryptionkey@0"
I20260806 02:16:24.315913 4374 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:16:24.318396 4392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:24.319689 4374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:24.319820 4374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "461b4acf249b49ce9fefa8ed4bf60e57"
format_stamp: "Formatted at 2026-08-06 02:16:24 on dist-test-slave-6k22"
server_key: "ba7587db325023b3922e62ef64f565f7"
server_key_iv: "de3e2a418bf5663717998c0b9e54bf2e"
server_key_version: "encryptionkey@0"
I20260806 02:16:24.319943 4374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:24.337836 4374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:24.340907 4374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:24.341113 4374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:24.341701 4374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:24.342530 4374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:24.342600 4374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:24.342674 4374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:24.342727 4374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:24.351660 4374 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:46021
I20260806 02:16:24.351723 4515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:46021 every 8 connection(s)
I20260806 02:16:24.352594 4374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:16:24.355746 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 4374
I20260806 02:16:24.355890 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:16:24.356235 18713 external_mini_cluster.cc:1428] Setting key 905fadf1187a0999b80448c54edf4fdd
Aug 06 02:16:24 dist-test-slave-6k22 krb5kdc[4114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982584, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
I20260806 02:16:24.367818 4202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.354393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:41321 (local address 127.18.70.126:42301)
0806 02:16:24.354531 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:24.354535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:24.355355 (+ 820us) server_negotiation.cc:407] Connection header received
0806 02:16:24.357131 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:24.357134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:24.357188 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:24.357268 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:24.358768 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.359282 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.360663 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.360923 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.361037 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.363852 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:24.363859 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:24.363866 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:24.363901 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:24.365610 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:24.366127 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:24.366133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:24.366136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:24.366232 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:24.366575 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:24.366582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:24.366586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:24.366852 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:24.366970 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:24.367438 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 02:16:24.367589 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 02:16:24.368580 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.354775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42301 (local address 127.18.70.66:41321)
0806 02:16:24.355211 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:24.355254 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:24.356917 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:24.357433 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:24.357441 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:24.357880 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:24.358601 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.358626 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.359424 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.359431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.360532 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.360542 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.361073 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.361078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.361257 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.361825 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:16:24.361841 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:24.363679 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:24.365732 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:24.365737 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:24.365749 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:24.366004 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:24.366334 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:24.366336 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:24.366337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:24.366465 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:24.366961 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:24.366966 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:24.367186 (+ 220us) client_negotiation.cc:769] Sending connection context
0806 02:16:24.367340 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 02:16:24.367645 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":116,"threads_started":1}
I20260806 02:16:24.369877 4516 heartbeater.cc:347] Connected to a master server at 127.18.70.126:42301
I20260806 02:16:24.370092 4516 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
I20260806 02:16:24.370635 4516 heartbeater.cc:511] Master 127.18.70.126:42301 requested a full tablet report, sending...
I20260806 02:16:24.371735 4161 ts_manager.cc:194] Registered new tserver with Master: 461b4acf249b49ce9fefa8ed4bf60e57 (127.18.70.66:46021)
I20260806 02:16:24.372589 4161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:41321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:16:24.386806 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42301
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:37415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:24.500186 4525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:24.500411 4525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:24.500481 4525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:24.504467 4525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:24.504518 4525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:24.504608 4525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:16:24.509770 4525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:37415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:42301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:24.510794 4525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:24.511823 4525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:24.521462 4530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:24.521476 4533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:24.521493 4531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:24.521890 4525 server_base.cc:1034] running on GCE node
I20260806 02:16:24.522310 4525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:24.523067 4525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:24.524231 4525 hybrid_clock.cc:648] HybridClock initialized: now 1785982584524211 us; error 29 us; skew 500 ppm
Aug 06 02:16:24 dist-test-slave-6k22 krb5kdc[4114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982584, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:24.527118 4525 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:16:24.528023 4525 webserver.cc:460] Webserver started at http://127.18.70.67:37337/ using document root <none> and password file <none>
I20260806 02:16:24.528587 4525 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:24.528719 4525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:24.528954 4525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:24.531014 4525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "3d8ec5663d7c4a2297bef564a31833da"
format_stamp: "Formatted at 2026-08-06 02:16:24 on dist-test-slave-6k22"
server_key: "c209bfc245ae83eb3ea533defa8d2128"
server_key_iv: "1a7e3b5280166ea24a52a2e49b33e56c"
server_key_version: "encryptionkey@0"
I20260806 02:16:24.531642 4525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "3d8ec5663d7c4a2297bef564a31833da"
format_stamp: "Formatted at 2026-08-06 02:16:24 on dist-test-slave-6k22"
server_key: "c209bfc245ae83eb3ea533defa8d2128"
server_key_iv: "1a7e3b5280166ea24a52a2e49b33e56c"
server_key_version: "encryptionkey@0"
I20260806 02:16:24.535319 4525 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:16:24.537737 4541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:24.539028 4525 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:16:24.539151 4525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "3d8ec5663d7c4a2297bef564a31833da"
format_stamp: "Formatted at 2026-08-06 02:16:24 on dist-test-slave-6k22"
server_key: "c209bfc245ae83eb3ea533defa8d2128"
server_key_iv: "1a7e3b5280166ea24a52a2e49b33e56c"
server_key_version: "encryptionkey@0"
I20260806 02:16:24.539258 4525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:24.586388 4525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:24.589354 4525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:24.589560 4525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:24.590170 4525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:24.590996 4525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:24.591064 4525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:24.591130 4525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:24.591166 4525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:24.599907 4525 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:44643
I20260806 02:16:24.599938 4661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:44643 every 8 connection(s)
I20260806 02:16:24.600840 4525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:16:24.603873 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 4525
I20260806 02:16:24.604008 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:16:24.604296 18713 external_mini_cluster.cc:1428] Setting key e82395e86f84a9c1148f19f4d0a70b02
Aug 06 02:16:24 dist-test-slave-6k22 krb5kdc[4114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982584, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:16:24.613978 4202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.602650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:49513 (local address 127.18.70.126:42301)
0806 02:16:24.602779 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:24.602782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:24.603656 (+ 874us) server_negotiation.cc:407] Connection header received
0806 02:16:24.605138 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:24.605141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:24.605196 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:24.605281 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:24.606440 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.606975 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.608189 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.608417 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.608526 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.610744 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:24.610750 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:24.610758 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:24.610780 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:24.611987 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:24.612501 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:24.612505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:24.612507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:24.612589 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:24.612930 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:24.612934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:24.612936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:24.613113 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:24.613222 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:24.613687 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 02:16:24.613812 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 02:16:24.614710 4664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.602998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:42301 (local address 127.18.70.67:49513)
0806 02:16:24.603481 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:24.603533 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:24.604930 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:24.605465 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:24.605473 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:24.605806 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:24.606301 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.606314 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.607102 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.607105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.608053 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.608062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.608567 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.608571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.608739 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.609228 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:16:24.609244 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:24.610584 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:24.612101 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:24.612107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:24.612121 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:24.612377 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:24.612683 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:24.612686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:24.612688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:24.612830 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:24.613226 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:24.613234 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:24.613459 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 02:16:24.613645 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 02:16:24.613945 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":111,"threads_started":1}
I20260806 02:16:24.615830 4662 heartbeater.cc:347] Connected to a master server at 127.18.70.126:42301
I20260806 02:16:24.616093 4662 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:24.616609 4662 heartbeater.cc:511] Master 127.18.70.126:42301 requested a full tablet report, sending...
I20260806 02:16:24.617745 4161 ts_manager.cc:194] Registered new tserver with Master: 3d8ec5663d7c4a2297bef564a31833da (127.18.70.67:44643)
I20260806 02:16:24.618165 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:16:24.618446 4161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:49513
I20260806 02:16:24.627245 4202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.620484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.18.70.126:42301)
0806 02:16:24.620608 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:24.620611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:24.620851 (+ 240us) server_negotiation.cc:407] Connection header received
0806 02:16:24.620949 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:24.620951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:24.620984 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:24.621044 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:24.621711 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.622167 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.622989 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.623212 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.623283 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.624687 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:24.624701 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:24.624703 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:24.624721 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:24.625852 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:24.626197 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:24.626199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:24.626200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:24.626265 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:24.626480 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:24.626482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:24.626483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:24.626655 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:24.626763 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:24.626955 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 02:16:24.627052 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:16:24.629784 4161 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:16:24.632690 4161 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:16:24.646847 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.641392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.18.70.66:46021)
0806 02:16:24.642036 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:24.642049 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:24.642072 (+ 23us) server_negotiation.cc:407] Connection header received
0806 02:16:24.642153 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:24.642157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:24.642356 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:24.642587 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:24.642842 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.643675 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.645469 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.646090 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.646224 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.646381 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:24.646586 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 02:16:24.646673 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":157,"threads_started":1}
I20260806 02:16:24.646836 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.640658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:46021 (local address 127.0.0.1:48604)
0806 02:16:24.640995 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:24.641041 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:24.641161 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:24.642501 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:24.642508 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:24.642535 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:24.642682 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.642700 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.643826 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.643829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.645333 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.645344 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.646217 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.646221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.646374 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.646421 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 02:16:24.646550 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 02:16:24.646672 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":63,"thread_start_us":103,"threads_started":1}
I20260806 02:16:24.647044 4678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.640735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:46387 (local address 127.0.0.1:43600)
0806 02:16:24.641559 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:24.641574 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:24.641665 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:24.642671 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:24.642674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:24.642691 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:24.642782 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.642805 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.643923 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.643929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.645279 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.645289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.646470 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.646474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.646559 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.646580 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 02:16:24.646643 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 02:16:24.646707 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"mutex_wait_us":74,"thread_start_us":100,"threads_started":1}
I20260806 02:16:24.647604 4681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.641481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43600 (local address 127.18.70.65:46387)
0806 02:16:24.641903 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:24.641920 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:24.641943 (+ 23us) server_negotiation.cc:407] Connection header received
0806 02:16:24.642037 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:24.642048 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:24.642276 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:24.642473 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:24.642936 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.643780 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.645396 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.646170 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.646270 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.646411 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:24.647353 (+ 942us) server_negotiation.cc:299] Negotiation successful
0806 02:16:24.647444 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":128,"threads_started":1}
I20260806 02:16:24.648419 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.641019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:44643 (local address 127.0.0.1:59400)
0806 02:16:24.641850 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:24.641863 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:24.641946 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:24.644245 (+ 2299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:24.644248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:24.644266 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:24.644359 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.644366 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.645665 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.645668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.646744 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.646752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.648044 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.648048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.648132 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.648154 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 02:16:24.648226 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 02:16:24.648291 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":2,"thread_start_us":85,"threads_started":1}
I20260806 02:16:24.648954 4680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.641468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59400 (local address 127.18.70.67:44643)
0806 02:16:24.643814 (+ 2346us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:24.643825 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:24.643846 (+ 21us) server_negotiation.cc:407] Connection header received
0806 02:16:24.643913 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:24.643918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:24.644095 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:24.644255 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:24.644461 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.645533 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.646885 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.647732 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.647851 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.647981 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:24.648679 (+ 698us) server_negotiation.cc:299] Negotiation successful
0806 02:16:24.648773 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2237,"thread_start_us":94,"threads_started":1}
I20260806 02:16:24.650523 4299 tablet_service.cc:1467] Processing CreateTablet for tablet 6b42ea1299e549d29c5dd7e021338d9c (DEFAULT_TABLE table=test-table [id=58c6c88b6cf542f98cb87c26b90a2d69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:24.650525 4446 tablet_service.cc:1467] Processing CreateTablet for tablet 6b42ea1299e549d29c5dd7e021338d9c (DEFAULT_TABLE table=test-table [id=58c6c88b6cf542f98cb87c26b90a2d69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:24.651062 4592 tablet_service.cc:1467] Processing CreateTablet for tablet 6b42ea1299e549d29c5dd7e021338d9c (DEFAULT_TABLE table=test-table [id=58c6c88b6cf542f98cb87c26b90a2d69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:24.651722 4446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b42ea1299e549d29c5dd7e021338d9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:24.651722 4299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b42ea1299e549d29c5dd7e021338d9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:24.651901 4592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b42ea1299e549d29c5dd7e021338d9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:24.658972 4682 tablet_bootstrap.cc:492] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da: Bootstrap starting.
I20260806 02:16:24.659706 4683 tablet_bootstrap.cc:492] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76: Bootstrap starting.
I20260806 02:16:24.661296 4682 tablet_bootstrap.cc:654] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:24.661842 4683 tablet_bootstrap.cc:654] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:24.662392 4682 log.cc:824] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:24.662590 4683 log.cc:824] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:24.663100 4684 tablet_bootstrap.cc:492] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57: Bootstrap starting.
I20260806 02:16:24.664731 4682 tablet_bootstrap.cc:492] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da: No bootstrap required, opened a new log
I20260806 02:16:24.664721 4683 tablet_bootstrap.cc:492] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76: No bootstrap required, opened a new log
I20260806 02:16:24.664927 4682 ts_tablet_manager.cc:1397] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 02:16:24.664927 4683 ts_tablet_manager.cc:1397] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 02:16:24.665263 4684 tablet_bootstrap.cc:654] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:24.665890 4684 log.cc:824] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:24.667754 4684 tablet_bootstrap.cc:492] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57: No bootstrap required, opened a new log
I20260806 02:16:24.667953 4684 ts_tablet_manager.cc:1397] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 02:16:24.669184 4682 raft_consensus.cc:348] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 46021 } } peers { permanent_uuid: "d7508c100b7d436596200d7aef62ee76" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 46387 } } peers { permanent_uuid: "3d8ec5663d7c4a2297bef564a31833da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 44643 } }
I20260806 02:16:24.669507 4682 raft_consensus.cc:374] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:24.669569 4682 raft_consensus.cc:729] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d8ec5663d7c4a2297bef564a31833da, State: Initialized, Role: FOLLOWER
I20260806 02:16:24.669486 4683 raft_consensus.cc:348] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 46021 } } peers { permanent_uuid: "d7508c100b7d436596200d7aef62ee76" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 46387 } } peers { permanent_uuid: "3d8ec5663d7c4a2297bef564a31833da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 44643 } }
I20260806 02:16:24.669751 4683 raft_consensus.cc:374] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:24.669829 4683 raft_consensus.cc:729] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7508c100b7d436596200d7aef62ee76, State: Initialized, Role: FOLLOWER
I20260806 02:16:24.670078 4682 consensus_queue.cc:260] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 46021 } } peers { permanent_uuid: "d7508c100b7d436596200d7aef62ee76" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 46387 } } peers { permanent_uuid: "3d8ec5663d7c4a2297bef564a31833da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 44643 } }
I20260806 02:16:24.670208 4683 consensus_queue.cc:260] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 46021 } } peers { permanent_uuid: "d7508c100b7d436596200d7aef62ee76" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 46387 } } peers { permanent_uuid: "3d8ec5663d7c4a2297bef564a31833da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 44643 } }
I20260806 02:16:24.670761 4662 heartbeater.cc:503] Master 127.18.70.126:42301 was elected leader, sending a full tablet report...
I20260806 02:16:24.671114 4682 ts_tablet_manager.cc:1428] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260806 02:16:24.671121 4683 ts_tablet_manager.cc:1428] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 02:16:24.671452 4369 heartbeater.cc:503] Master 127.18.70.126:42301 was elected leader, sending a full tablet report...
I20260806 02:16:24.672475 4684 raft_consensus.cc:348] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 46021 } } peers { permanent_uuid: "d7508c100b7d436596200d7aef62ee76" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 46387 } } peers { permanent_uuid: "3d8ec5663d7c4a2297bef564a31833da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 44643 } }
I20260806 02:16:24.672793 4684 raft_consensus.cc:374] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:24.672870 4684 raft_consensus.cc:729] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 461b4acf249b49ce9fefa8ed4bf60e57, State: Initialized, Role: FOLLOWER
I20260806 02:16:24.673282 4684 consensus_queue.cc:260] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 46021 } } peers { permanent_uuid: "d7508c100b7d436596200d7aef62ee76" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 46387 } } peers { permanent_uuid: "3d8ec5663d7c4a2297bef564a31833da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 44643 } }
I20260806 02:16:24.674263 4516 heartbeater.cc:503] Master 127.18.70.126:42301 was elected leader, sending a full tablet report...
I20260806 02:16:24.674394 4684 ts_tablet_manager.cc:1428] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260806 02:16:24.851816 4663 tablet.cc:2369] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 02:16:24.853736 4517 tablet.cc:2369] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 02:16:24.883049 4370 tablet.cc:2369] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:16:24.979322 4690 raft_consensus.cc:482] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:16:24.979588 4690 raft_consensus.cc:504] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 46021 } } peers { permanent_uuid: "d7508c100b7d436596200d7aef62ee76" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 46387 } } peers { permanent_uuid: "3d8ec5663d7c4a2297bef564a31833da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 44643 } }
I20260806 02:16:24.980762 4690 leader_election.cc:288] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7508c100b7d436596200d7aef62ee76 (127.18.70.65:46387), 3d8ec5663d7c4a2297bef564a31833da (127.18.70.67:44643)
I20260806 02:16:24.984838 4680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.981152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:41385 (local address 127.18.70.67:44643)
0806 02:16:24.981285 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:24.981290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:24.981611 (+ 321us) server_negotiation.cc:407] Connection header received
0806 02:16:24.981742 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:24.981744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:24.981788 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:24.981881 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:24.982172 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.982637 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.983814 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.984221 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.984339 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.984396 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:24.984604 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 02:16:24.984674 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:16:24.984846 4692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.981008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:46387 (local address 127.18.70.66:47507)
0806 02:16:24.981357 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:24.981370 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:24.981474 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:24.981739 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:24.981741 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:24.981759 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:24.981900 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.981907 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.982699 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.982703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.983692 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.983702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.984390 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.984393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.984496 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.984514 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 02:16:24.984624 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 02:16:24.984703 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":19,"thread_start_us":97,"threads_started":1}
I20260806 02:16:24.984853 4691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.981018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:44643 (local address 127.18.70.66:41385)
0806 02:16:24.981500 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:24.981517 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:24.981618 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:24.981902 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:24.981904 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:24.981918 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:24.982061 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.982067 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.982775 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.982778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.983682 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:24.983697 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.984349 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.984352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:24.984452 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.984475 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 02:16:24.984546 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 02:16:24.984640 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":17,"thread_start_us":77,"threads_started":1}
I20260806 02:16:24.984860 4681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:24.981156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:47507 (local address 127.18.70.65:46387)
0806 02:16:24.981312 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:24.981316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:24.981458 (+ 142us) server_negotiation.cc:407] Connection header received
0806 02:16:24.981585 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:24.981590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:24.981635 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:24.981714 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:24.982023 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.982568 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.983844 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:24.984238 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:24.984371 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:24.984423 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:24.984662 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 02:16:24.984749 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 02:16:24.985460 4612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b42ea1299e549d29c5dd7e021338d9c" candidate_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d8ec5663d7c4a2297bef564a31833da" is_pre_election: true
I20260806 02:16:24.985467 4319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b42ea1299e549d29c5dd7e021338d9c" candidate_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7508c100b7d436596200d7aef62ee76" is_pre_election: true
I20260806 02:16:24.985745 4612 raft_consensus.cc:2454] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 461b4acf249b49ce9fefa8ed4bf60e57 in term 0.
I20260806 02:16:24.985772 4319 raft_consensus.cc:2454] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 461b4acf249b49ce9fefa8ed4bf60e57 in term 0.
I20260806 02:16:24.986260 4396 leader_election.cc:302] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d8ec5663d7c4a2297bef564a31833da, 461b4acf249b49ce9fefa8ed4bf60e57; no voters:
I20260806 02:16:24.986531 4690 raft_consensus.cc:2781] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:16:24.986622 4690 raft_consensus.cc:482] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:16:24.986685 4690 raft_consensus.cc:3037] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:24.987784 4690 raft_consensus.cc:504] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 46021 } } peers { permanent_uuid: "d7508c100b7d436596200d7aef62ee76" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 46387 } } peers { permanent_uuid: "3d8ec5663d7c4a2297bef564a31833da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 44643 } }
I20260806 02:16:24.988111 4690 leader_election.cc:288] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [CANDIDATE]: Term 1 election: Requested vote from peers d7508c100b7d436596200d7aef62ee76 (127.18.70.65:46387), 3d8ec5663d7c4a2297bef564a31833da (127.18.70.67:44643)
I20260806 02:16:24.988476 4319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b42ea1299e549d29c5dd7e021338d9c" candidate_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7508c100b7d436596200d7aef62ee76"
I20260806 02:16:24.988518 4612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b42ea1299e549d29c5dd7e021338d9c" candidate_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d8ec5663d7c4a2297bef564a31833da"
I20260806 02:16:24.988583 4319 raft_consensus.cc:3037] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:24.988641 4612 raft_consensus.cc:3037] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:24.989594 4319 raft_consensus.cc:2454] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 461b4acf249b49ce9fefa8ed4bf60e57 in term 1.
I20260806 02:16:24.989610 4612 raft_consensus.cc:2454] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 461b4acf249b49ce9fefa8ed4bf60e57 in term 1.
I20260806 02:16:24.989938 4396 leader_election.cc:302] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d8ec5663d7c4a2297bef564a31833da, 461b4acf249b49ce9fefa8ed4bf60e57; no voters:
I20260806 02:16:24.990103 4690 raft_consensus.cc:2781] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:24.990310 4690 raft_consensus.cc:686] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [term 1 LEADER]: Becoming Leader. State: Replica: 461b4acf249b49ce9fefa8ed4bf60e57, State: Running, Role: LEADER
I20260806 02:16:24.990653 4690 consensus_queue.cc:237] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 46021 } } peers { permanent_uuid: "d7508c100b7d436596200d7aef62ee76" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 46387 } } peers { permanent_uuid: "3d8ec5663d7c4a2297bef564a31833da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 44643 } }
I20260806 02:16:24.993870 4159 catalog_manager.cc:5495] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 461b4acf249b49ce9fefa8ed4bf60e57 (127.18.70.66). New cstate: current_term: 1 leader_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "461b4acf249b49ce9fefa8ed4bf60e57" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 46021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7508c100b7d436596200d7aef62ee76" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 46387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d8ec5663d7c4a2297bef564a31833da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 44643 } health_report { overall_health: UNKNOWN } } }
I20260806 02:16:25.011348 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.008137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48618 (local address 127.18.70.66:46021)
0806 02:16:25.008253 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:25.008257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:25.008271 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:16:25.008429 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:25.008431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:25.008481 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:25.008564 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:16:25.008775 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.009255 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.010044 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.010253 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.010326 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.010579 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:16:25.010989 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:16:25.011071 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:25.011130 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 02:16:25.011173 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 02:16:25.017354 4319 raft_consensus.cc:1261] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76 [term 1 FOLLOWER]: Refusing update from remote peer 461b4acf249b49ce9fefa8ed4bf60e57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:16:25.017352 4612 raft_consensus.cc:1261] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da [term 1 FOLLOWER]: Refusing update from remote peer 461b4acf249b49ce9fefa8ed4bf60e57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:16:25.018065 4690 consensus_queue.cc:1035] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d8ec5663d7c4a2297bef564a31833da" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 44643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:16:25.018230 4693 consensus_queue.cc:1035] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7508c100b7d436596200d7aef62ee76" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 46387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:16:25.026224 4698 mvcc.cc:205] Tried to move back new op lower bound from 7315384668224020480 to 7315384668124508160. Current Snapshot: MvccSnapshot[applied={T|T < 7315384668224020480}]
I20260806 02:16:25.035519 4160 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:16:25.035887 4160 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:16:25.038380 4160 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4c91dc93342e420bbc40d90d3838d4a2: Sending DeleteTablet for 3 replicas of tablet 6b42ea1299e549d29c5dd7e021338d9c
I20260806 02:16:25.039106 4446 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b42ea1299e549d29c5dd7e021338d9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:25 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:48604
I20260806 02:16:25.039222 4299 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b42ea1299e549d29c5dd7e021338d9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:25 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:43600
I20260806 02:16:25.039621 4707 tablet_replica.cc:330] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57: stopping tablet replica
I20260806 02:16:25.039638 4708 tablet_replica.cc:330] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76: stopping tablet replica
I20260806 02:16:25.039794 4592 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b42ea1299e549d29c5dd7e021338d9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:25 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:59400
I20260806 02:16:25.040014 4708 raft_consensus.cc:2229] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:16:25.040032 4707 raft_consensus.cc:2229] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:16:25.040346 4708 raft_consensus.cc:2258] T 6b42ea1299e549d29c5dd7e021338d9c P d7508c100b7d436596200d7aef62ee76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:16:25.040537 4707 raft_consensus.cc:2258] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:16:25.040928 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 4227
I20260806 02:16:25.042461 4707 ts_tablet_manager.cc:1905] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:16:25.045257 4707 ts_tablet_manager.cc:1918] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:16:25.045424 4707 log.cc:1196] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/6b42ea1299e549d29c5dd7e021338d9c
I20260806 02:16:25.045785 4707 ts_tablet_manager.cc:1939] T 6b42ea1299e549d29c5dd7e021338d9c P 461b4acf249b49ce9fefa8ed4bf60e57: Deleting consensus metadata
I20260806 02:16:25.045969 4709 tablet_replica.cc:330] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da: stopping tablet replica
I20260806 02:16:25.046306 4709 raft_consensus.cc:2229] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:16:25.046609 4709 raft_consensus.cc:2258] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:16:25.047122 4146 catalog_manager.cc:4841] TS 461b4acf249b49ce9fefa8ed4bf60e57 (127.18.70.66:46021): tablet 6b42ea1299e549d29c5dd7e021338d9c (table test-table [id=58c6c88b6cf542f98cb87c26b90a2d69]) successfully deleted
W20260806 02:16:25.053099 4145 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:46387 (error 108)
I20260806 02:16:25.054060 4709 ts_tablet_manager.cc:1905] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:16:25.054155 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.053836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:46387 (local address 127.0.0.1:43616)
0806 02:16:25.053966 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:25.054052 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:46387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 02:16:25.054267 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 4374
W20260806 02:16:25.056442 4145 catalog_manager.cc:4584] TS d7508c100b7d436596200d7aef62ee76 (127.18.70.65:46387): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b42ea1299e549d29c5dd7e021338d9c: Network error: Client connection negotiation failed: client connection to 127.18.70.65:46387: connect: Connection refused (error 111)
I20260806 02:16:25.056823 4709 ts_tablet_manager.cc:1918] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:16:25.056926 4709 log.cc:1196] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/6b42ea1299e549d29c5dd7e021338d9c
I20260806 02:16:25.057250 4709 ts_tablet_manager.cc:1939] T 6b42ea1299e549d29c5dd7e021338d9c P 3d8ec5663d7c4a2297bef564a31833da: Deleting consensus metadata
I20260806 02:16:25.058542 4147 catalog_manager.cc:4841] TS 3d8ec5663d7c4a2297bef564a31833da (127.18.70.67:44643): tablet 6b42ea1299e549d29c5dd7e021338d9c (table test-table [id=58c6c88b6cf542f98cb87c26b90a2d69]) successfully deleted
I20260806 02:16:25.062601 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 4525
I20260806 02:16:25.069344 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 4127
2026-08-06T02:16:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2681 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T02:16:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:25Z Disabled control of system clock
I20260806 02:16:25.101224 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:35475
--webserver_interface=127.18.70.126
--webserver_port=0
--builtin_ntp_servers=127.18.70.84:38093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:35475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 02:16:25.207336 4716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:25.207572 4716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:25.207656 4716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:25.211484 4716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:25.211532 4716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:25.211547 4716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:25.211580 4716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:25.211623 4716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:25.216236 4716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:38093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:35475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:35475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:25.217204 4716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:25.218536 4716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:25.226614 4722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:25.226629 4721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:25.226632 4724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:25.227137 4716 server_base.cc:1034] running on GCE node
I20260806 02:16:25.227638 4716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:25.228618 4716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:25.229789 4716 hybrid_clock.cc:648] HybridClock initialized: now 1785982585229756 us; error 35 us; skew 500 ppm
I20260806 02:16:25.231942 4716 webserver.cc:460] Webserver started at http://127.18.70.126:37745/ using document root <none> and password file <none>
I20260806 02:16:25.232534 4716 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:25.232625 4716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:25.232846 4716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:25.234788 4716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "c03fda0efe2849fc980df3cda7a99d8f"
format_stamp: "Formatted at 2026-08-06 02:16:25 on dist-test-slave-6k22"
server_key: "89443fadb8b902f6e2c047f0d40c0484"
server_key_iv: "96da8642240257ca6b7ba593da96183d"
server_key_version: "encryptionkey@0"
I20260806 02:16:25.235316 4716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "c03fda0efe2849fc980df3cda7a99d8f"
format_stamp: "Formatted at 2026-08-06 02:16:25 on dist-test-slave-6k22"
server_key: "89443fadb8b902f6e2c047f0d40c0484"
server_key_iv: "96da8642240257ca6b7ba593da96183d"
server_key_version: "encryptionkey@0"
I20260806 02:16:25.238849 4716 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 02:16:25.241163 4730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:25.242282 4716 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:16:25.242419 4716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "c03fda0efe2849fc980df3cda7a99d8f"
format_stamp: "Formatted at 2026-08-06 02:16:25 on dist-test-slave-6k22"
server_key: "89443fadb8b902f6e2c047f0d40c0484"
server_key_iv: "96da8642240257ca6b7ba593da96183d"
server_key_version: "encryptionkey@0"
I20260806 02:16:25.242525 4716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:25.254529 4716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:25.255117 4716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:25.255300 4716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:25.262229 4716 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:35475
I20260806 02:16:25.262245 4786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:35475 every 8 connection(s)
I20260806 02:16:25.263114 4716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:25.266112 4787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:25.266960 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 4716
I20260806 02:16:25.267095 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:25.267395 18713 external_mini_cluster.cc:1428] Setting key a36e1587929328dcc8ea6ddafe262eae
I20260806 02:16:25.271982 4787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f: Bootstrap starting.
I20260806 02:16:25.274183 4787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:25.274935 4787 log.cc:824] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:25.276432 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.268863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57642 (local address 127.18.70.126:35475)
0806 02:16:25.269454 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:25.269469 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:25.269503 (+ 34us) server_negotiation.cc:407] Connection header received
0806 02:16:25.270150 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:25.270166 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:25.270528 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:25.270860 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:25.271313 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.272391 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.273245 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.273705 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.273867 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.274731 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:25.274737 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:25.274749 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:25.275162 (+ 413us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:25.275217 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:25.275237 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:25.275339 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:25.275491 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 02:16:25.275758 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":117,"threads_started":1}
I20260806 02:16:25.276940 4787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f: No bootstrap required, opened a new log
I20260806 02:16:25.280364 4787 raft_consensus.cc:348] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03fda0efe2849fc980df3cda7a99d8f" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35475 } }
I20260806 02:16:25.280566 4787 raft_consensus.cc:374] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:25.280642 4787 raft_consensus.cc:729] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c03fda0efe2849fc980df3cda7a99d8f, State: Initialized, Role: FOLLOWER
I20260806 02:16:25.281100 4787 consensus_queue.cc:260] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03fda0efe2849fc980df3cda7a99d8f" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35475 } }
I20260806 02:16:25.281239 4787 raft_consensus.cc:388] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:25.281304 4787 raft_consensus.cc:482] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:25.281430 4787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:25.282270 4787 raft_consensus.cc:504] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03fda0efe2849fc980df3cda7a99d8f" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35475 } }
I20260806 02:16:25.282605 4787 leader_election.cc:302] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c03fda0efe2849fc980df3cda7a99d8f; no voters:
I20260806 02:16:25.282912 4787 leader_election.cc:288] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:25.283031 4793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:25.283267 4793 raft_consensus.cc:686] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [term 1 LEADER]: Becoming Leader. State: Replica: c03fda0efe2849fc980df3cda7a99d8f, State: Running, Role: LEADER
I20260806 02:16:25.283541 4793 consensus_queue.cc:237] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03fda0efe2849fc980df3cda7a99d8f" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35475 } }
I20260806 02:16:25.284194 4787 sys_catalog.cc:564] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:25.285336 4794 sys_catalog.cc:455] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c03fda0efe2849fc980df3cda7a99d8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03fda0efe2849fc980df3cda7a99d8f" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35475 } } }
I20260806 02:16:25.285336 4795 sys_catalog.cc:455] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c03fda0efe2849fc980df3cda7a99d8f. Latest consensus state: current_term: 1 leader_uuid: "c03fda0efe2849fc980df3cda7a99d8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03fda0efe2849fc980df3cda7a99d8f" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35475 } } }
I20260806 02:16:25.285487 4795 sys_catalog.cc:458] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:25.285487 4794 sys_catalog.cc:458] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:25.285909 4803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:25.288578 4803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:25.296850 4803 catalog_manager.cc:1347] Generated new cluster ID: 7e53fd8bc3fd4f59b3c65c623e24d274
I20260806 02:16:25.296917 4803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:25.311302 4803 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:25.312219 4803 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:25.320838 4803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f: Generated new TSK 0
I20260806 02:16:25.321535 4803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:25.324409 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35475
--builtin_ntp_servers=127.18.70.84:38093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 02:16:25.437521 4814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:25.437747 4814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:25.437783 4814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:25.441648 4814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:25.441695 4814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:25.441747 4814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:16:25.446441 4814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:38093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:25.447335 4814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:25.448626 4814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:25.458163 4821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:25.458194 4820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:25.458391 4814 server_base.cc:1034] running on GCE node
W20260806 02:16:25.458564 4823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:25.458946 4814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:25.459631 4814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:25.460815 4814 hybrid_clock.cc:648] HybridClock initialized: now 1785982585460785 us; error 52 us; skew 500 ppm
I20260806 02:16:25.462535 4814 webserver.cc:460] Webserver started at http://127.18.70.65:41595/ using document root <none> and password file <none>
I20260806 02:16:25.463106 4814 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:25.463204 4814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:25.463430 4814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:25.465468 4814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "51a1c034796345d5a7ef8336561197dd"
format_stamp: "Formatted at 2026-08-06 02:16:25 on dist-test-slave-6k22"
server_key: "2833026aba1fb6429010452aa1baec11"
server_key_iv: "08523e0703bc0c7b899e040fcd6da5d2"
server_key_version: "encryptionkey@0"
I20260806 02:16:25.466009 4814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "51a1c034796345d5a7ef8336561197dd"
format_stamp: "Formatted at 2026-08-06 02:16:25 on dist-test-slave-6k22"
server_key: "2833026aba1fb6429010452aa1baec11"
server_key_iv: "08523e0703bc0c7b899e040fcd6da5d2"
server_key_version: "encryptionkey@0"
I20260806 02:16:25.469655 4814 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:16:25.472045 4829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:25.473321 4814 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:25.473450 4814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "51a1c034796345d5a7ef8336561197dd"
format_stamp: "Formatted at 2026-08-06 02:16:25 on dist-test-slave-6k22"
server_key: "2833026aba1fb6429010452aa1baec11"
server_key_iv: "08523e0703bc0c7b899e040fcd6da5d2"
server_key_version: "encryptionkey@0"
I20260806 02:16:25.473557 4814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:25.485262 4814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:25.485954 4814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:25.486145 4814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:25.486799 4814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:25.487775 4814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:25.487843 4814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:25.487905 4814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:25.487954 4814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:25.497352 4814 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:35561
I20260806 02:16:25.497391 4950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:35561 every 8 connection(s)
I20260806 02:16:25.498271 4814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:16:25.501296 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 4814
I20260806 02:16:25.501444 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:16:25.501752 18713 external_mini_cluster.cc:1428] Setting key 0219284090359c68ba3a6f008b90c63b
I20260806 02:16:25.503924 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35475
--builtin_ntp_servers=127.18.70.84:38093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 02:16:25.508550 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.500037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:47635 (local address 127.18.70.126:35475)
0806 02:16:25.500179 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:25.500183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:25.501238 (+ 1055us) server_negotiation.cc:407] Connection header received
0806 02:16:25.503017 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:25.503020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:25.503091 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:25.503178 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:25.504260 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.504811 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.506360 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.506669 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.506789 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.507960 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:25.507966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:25.507972 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:25.507990 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:25.508017 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:25.508030 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:25.508113 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:25.508259 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 02:16:25.508336 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 02:16:25.509245 4953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.500426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35475 (local address 127.18.70.65:47635)
0806 02:16:25.501090 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:25.501129 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:25.502803 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:25.503392 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:25.503402 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:25.503496 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:25.504082 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:25.504104 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.504946 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.504950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:25.506230 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:25.506243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.506844 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.506851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:25.507033 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.507686 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:16:25.507688 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:25.507737 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:16:25.507798 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:25.508118 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:25.508125 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:25.508130 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 02:16:25.508226 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 02:16:25.508433 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":124,"threads_started":1}
I20260806 02:16:25.510479 4951 heartbeater.cc:347] Connected to a master server at 127.18.70.126:35475
I20260806 02:16:25.510705 4951 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:25.511202 4951 heartbeater.cc:511] Master 127.18.70.126:35475 requested a full tablet report, sending...
I20260806 02:16:25.512704 4748 ts_manager.cc:194] Registered new tserver with Master: 51a1c034796345d5a7ef8336561197dd (127.18.70.65:35561)
I20260806 02:16:25.513857 4748 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.70.65:47635
W20260806 02:16:25.618476 4955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:25.618717 4955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:25.618753 4955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:25.622825 4955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:25.622871 4955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:25.622921 4955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:16:25.627765 4955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:38093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:25.628801 4955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:25.630123 4955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:25.639796 4961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:25.639796 4964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:25.639989 4955 server_base.cc:1034] running on GCE node
W20260806 02:16:25.639822 4960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:25.640461 4955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:25.641059 4955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:25.642223 4955 hybrid_clock.cc:648] HybridClock initialized: now 1785982585642207 us; error 28 us; skew 500 ppm
I20260806 02:16:25.644034 4955 webserver.cc:460] Webserver started at http://127.18.70.66:34289/ using document root <none> and password file <none>
I20260806 02:16:25.644586 4955 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:25.644644 4955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:25.644816 4955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:25.646974 4955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "0f82b3d4504a4bd39c034262a058b0a0"
format_stamp: "Formatted at 2026-08-06 02:16:25 on dist-test-slave-6k22"
server_key: "54b3a6e9d8a725c63cb66b15bd443e67"
server_key_iv: "6c6a8e21299c6c2ffd2193ec8697f1d3"
server_key_version: "encryptionkey@0"
I20260806 02:16:25.647490 4955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "0f82b3d4504a4bd39c034262a058b0a0"
format_stamp: "Formatted at 2026-08-06 02:16:25 on dist-test-slave-6k22"
server_key: "54b3a6e9d8a725c63cb66b15bd443e67"
server_key_iv: "6c6a8e21299c6c2ffd2193ec8697f1d3"
server_key_version: "encryptionkey@0"
I20260806 02:16:25.651012 4955 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:16:25.653532 4971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:25.654739 4955 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:16:25.654832 4955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "0f82b3d4504a4bd39c034262a058b0a0"
format_stamp: "Formatted at 2026-08-06 02:16:25 on dist-test-slave-6k22"
server_key: "54b3a6e9d8a725c63cb66b15bd443e67"
server_key_iv: "6c6a8e21299c6c2ffd2193ec8697f1d3"
server_key_version: "encryptionkey@0"
I20260806 02:16:25.654914 4955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:25.667744 4955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:25.668303 4955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:25.668435 4955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:25.668928 4955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:25.669760 4955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:25.669806 4955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:25.669842 4955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:25.669857 4955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:25.678929 4955 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:37067
I20260806 02:16:25.679023 5098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:37067 every 8 connection(s)
I20260806 02:16:25.679843 4955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:16:25.681955 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 4955
I20260806 02:16:25.682098 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:16:25.682468 18713 external_mini_cluster.cc:1428] Setting key 7e998cc3f28d0fec169c413f976e144d
I20260806 02:16:25.684785 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35475
--builtin_ntp_servers=127.18.70.84:38093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 02:16:25.690160 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.681587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:50403 (local address 127.18.70.126:35475)
0806 02:16:25.681760 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:25.681765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:25.682934 (+ 1169us) server_negotiation.cc:407] Connection header received
0806 02:16:25.684634 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:25.684637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:25.684704 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:25.684778 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:25.686001 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.686779 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.687987 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.688375 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.688496 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.689555 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:25.689562 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:25.689566 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:25.689581 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:25.689605 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:25.689615 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:25.689698 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:25.689857 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 02:16:25.689957 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:16:25.690855 5101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.681947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35475 (local address 127.18.70.66:50403)
0806 02:16:25.682768 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:25.682820 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:25.684420 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:25.685008 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:25.685019 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:25.685125 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:25.685814 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:25.685836 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.686914 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.686917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:25.687870 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:25.687880 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.688520 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.688523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:25.688661 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.689299 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:16:25.689301 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:25.689339 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:16:25.689398 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:25.689710 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:25.689718 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:25.689723 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 02:16:25.689824 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 02:16:25.690058 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":169,"threads_started":1}
I20260806 02:16:25.692020 5099 heartbeater.cc:347] Connected to a master server at 127.18.70.126:35475
I20260806 02:16:25.692286 5099 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:25.692916 5099 heartbeater.cc:511] Master 127.18.70.126:35475 requested a full tablet report, sending...
I20260806 02:16:25.694070 4748 ts_manager.cc:194] Registered new tserver with Master: 0f82b3d4504a4bd39c034262a058b0a0 (127.18.70.66:37067)
I20260806 02:16:25.694625 4748 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.70.66:50403
W20260806 02:16:25.802268 5102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:25.802510 5102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:25.802543 5102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:25.806703 5102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:25.806754 5102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:25.806835 5102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:16:25.811489 5102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:38093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:25.812543 5102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:25.813947 5102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:25.824215 5109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:25.824226 5108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:25.824492 5102 server_base.cc:1034] running on GCE node
W20260806 02:16:25.824769 5111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:25.825271 5102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:25.826139 5102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:25.827346 5102 hybrid_clock.cc:648] HybridClock initialized: now 1785982585827316 us; error 37 us; skew 500 ppm
I20260806 02:16:25.829869 5102 webserver.cc:460] Webserver started at http://127.18.70.67:43757/ using document root <none> and password file <none>
I20260806 02:16:25.830678 5102 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:25.830780 5102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:25.831065 5102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:25.834331 5102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "0243f20817f844c3a6278a42fda5ef2d"
format_stamp: "Formatted at 2026-08-06 02:16:25 on dist-test-slave-6k22"
server_key: "534d2f5a2d1ef8b9a60bf4cccd67a407"
server_key_iv: "4c42dc527395d6638af9aa4243137a6f"
server_key_version: "encryptionkey@0"
I20260806 02:16:25.835106 5102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "0243f20817f844c3a6278a42fda5ef2d"
format_stamp: "Formatted at 2026-08-06 02:16:25 on dist-test-slave-6k22"
server_key: "534d2f5a2d1ef8b9a60bf4cccd67a407"
server_key_iv: "4c42dc527395d6638af9aa4243137a6f"
server_key_version: "encryptionkey@0"
I20260806 02:16:25.840570 5102 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 02:16:25.844218 5117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:25.846007 5102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:16:25.846139 5102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "0243f20817f844c3a6278a42fda5ef2d"
format_stamp: "Formatted at 2026-08-06 02:16:25 on dist-test-slave-6k22"
server_key: "534d2f5a2d1ef8b9a60bf4cccd67a407"
server_key_iv: "4c42dc527395d6638af9aa4243137a6f"
server_key_version: "encryptionkey@0"
I20260806 02:16:25.846266 5102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:25.872522 5102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:25.873168 5102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:25.873364 5102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:25.873929 5102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:25.874843 5102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:25.874927 5102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:25.874992 5102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:25.875041 5102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:25.883868 5102 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:40653
I20260806 02:16:25.883932 5239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:40653 every 8 connection(s)
I20260806 02:16:25.884807 5102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:16:25.892364 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 5102
I20260806 02:16:25.892493 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:16:25.892799 18713 external_mini_cluster.cc:1428] Setting key 796705700734d2938c21dee6e74d8e2d
I20260806 02:16:25.893924 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.886714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:58357 (local address 127.18.70.126:35475)
0806 02:16:25.886887 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:25.886892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:25.888021 (+ 1129us) server_negotiation.cc:407] Connection header received
0806 02:16:25.889369 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:25.889372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:25.889418 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:25.889492 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:25.890332 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.890846 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.891991 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.892200 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.892303 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.893443 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:25.893446 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:25.893449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:25.893460 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:25.893485 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:25.893493 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:25.893567 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:25.893734 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 02:16:25.893790 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 02:16:25.894702 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.887196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35475 (local address 127.18.70.67:58357)
0806 02:16:25.887886 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:25.887925 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:25.889180 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:25.889655 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:25.889662 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:25.889739 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:25.890212 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:25.890223 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.890950 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.890952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:25.891892 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:25.891902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.892354 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.892358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:25.892569 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.893194 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:16:25.893195 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:25.893234 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:16:25.893290 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:25.893568 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:25.893580 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:25.893588 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 02:16:25.893718 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 02:16:25.893980 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":118,"threads_started":1}
I20260806 02:16:25.895743 5240 heartbeater.cc:347] Connected to a master server at 127.18.70.126:35475
I20260806 02:16:25.895999 5240 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:25.896489 5240 heartbeater.cc:511] Master 127.18.70.126:35475 requested a full tablet report, sending...
I20260806 02:16:25.897521 4748 ts_manager.cc:194] Registered new tserver with Master: 0243f20817f844c3a6278a42fda5ef2d (127.18.70.67:40653)
I20260806 02:16:25.898059 4748 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.70.67:58357
I20260806 02:16:25.906625 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:16:25.912634 4791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.908955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.18.70.126:35475)
0806 02:16:25.909066 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:25.909070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:25.909318 (+ 248us) server_negotiation.cc:407] Connection header received
0806 02:16:25.909420 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:25.909422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:25.909457 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:25.909521 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:25.909847 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.910311 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.911168 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.911370 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.911438 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.912246 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:25.912249 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:25.912251 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:25.912261 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:25.912279 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:25.912286 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:25.912354 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:25.912496 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 02:16:25.912540 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 02:16:25.915186 4748 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:16:25.918164 4748 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:16:25.932080 5254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.926207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:40653 (local address 127.0.0.1:51814)
0806 02:16:25.926682 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:25.926729 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:25.926824 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:25.927646 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:25.927655 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:25.927677 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:25.927783 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:25.927787 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.929142 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.929148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:25.930403 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:25.930415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.931502 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.931506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:25.931633 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.931677 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 02:16:25.931762 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 02:16:25.931903 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":68,"threads_started":1}
I20260806 02:16:25.932144 5258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.927108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41470 (local address 127.18.70.65:35561)
0806 02:16:25.927673 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:25.927687 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:25.927721 (+ 34us) server_negotiation.cc:407] Connection header received
0806 02:16:25.927801 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:25.927806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:25.927994 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:25.928144 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:25.928328 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.929460 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.930872 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.931498 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.931647 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.931852 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:25.931901 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 02:16:25.931991 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":100,"threads_started":1}
I20260806 02:16:25.932075 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.926919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35561 (local address 127.0.0.1:41470)
0806 02:16:25.927468 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:25.927482 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:25.927587 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:25.928135 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:25.928138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:25.928155 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:25.928234 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:25.928239 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.929590 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.929594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:25.930750 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:25.930759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.931649 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.931653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:25.931751 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.931777 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 02:16:25.931843 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 02:16:25.931920 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":34,"thread_start_us":82,"threads_started":1}
I20260806 02:16:25.932490 5253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.926311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:37067 (local address 127.0.0.1:49446)
0806 02:16:25.927314 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:25.927327 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:25.927404 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:25.928465 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:25.928467 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:25.928485 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:25.928567 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:25.928572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.929840 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.929845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:25.931011 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:25.931021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.932103 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.932109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:25.932205 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.932239 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 02:16:25.932292 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 02:16:25.932366 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":22,"thread_start_us":88,"threads_started":1}
I20260806 02:16:25.933019 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.926331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51814 (local address 127.18.70.67:40653)
0806 02:16:25.927116 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:25.927127 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:25.927152 (+ 25us) server_negotiation.cc:407] Connection header received
0806 02:16:25.927232 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:25.927238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:25.927452 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:25.927941 (+ 489us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:25.928057 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.929038 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.930538 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.931353 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.932144 (+ 791us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.932721 (+ 577us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:25.932770 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 02:16:25.932868 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":150,"threads_started":1}
I20260806 02:16:25.933223 5257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:25.927001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49446 (local address 127.18.70.66:37067)
0806 02:16:25.928037 (+ 1036us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:25.928053 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:25.928075 (+ 22us) server_negotiation.cc:407] Connection header received
0806 02:16:25.928147 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:25.928151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:25.928328 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:25.928479 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:25.928661 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.929732 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.931149 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:25.931966 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:25.932752 (+ 786us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:25.932918 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:25.932970 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 02:16:25.933063 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":929,"thread_start_us":105,"threads_started":1}
I20260806 02:16:25.935111 4881 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3bc377c2f8485a9b160fe4e10034c9 (DEFAULT_TABLE table=test-table [id=2a5ccb46f96349d4b548f8ec0cb6902b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:25.936091 4881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3bc377c2f8485a9b160fe4e10034c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:25.935662 5024 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3bc377c2f8485a9b160fe4e10034c9 (DEFAULT_TABLE table=test-table [id=2a5ccb46f96349d4b548f8ec0cb6902b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:25.935662 5168 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3bc377c2f8485a9b160fe4e10034c9 (DEFAULT_TABLE table=test-table [id=2a5ccb46f96349d4b548f8ec0cb6902b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:25.936668 5024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3bc377c2f8485a9b160fe4e10034c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:25.936668 5168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3bc377c2f8485a9b160fe4e10034c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:25.942695 5259 tablet_bootstrap.cc:492] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd: Bootstrap starting.
I20260806 02:16:25.943490 5260 tablet_bootstrap.cc:492] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0: Bootstrap starting.
I20260806 02:16:25.944346 5261 tablet_bootstrap.cc:492] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d: Bootstrap starting.
I20260806 02:16:25.945127 5259 tablet_bootstrap.cc:654] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:25.945645 5260 tablet_bootstrap.cc:654] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:25.946043 5259 log.cc:824] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:25.946074 5261 tablet_bootstrap.cc:654] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:25.946434 5260 log.cc:824] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:25.946942 5261 log.cc:824] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:25.948212 5259 tablet_bootstrap.cc:492] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd: No bootstrap required, opened a new log
I20260806 02:16:25.948414 5259 ts_tablet_manager.cc:1397] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260806 02:16:25.948520 5260 tablet_bootstrap.cc:492] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0: No bootstrap required, opened a new log
I20260806 02:16:25.948697 5260 ts_tablet_manager.cc:1397] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 02:16:25.948837 5261 tablet_bootstrap.cc:492] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d: No bootstrap required, opened a new log
I20260806 02:16:25.949015 5261 ts_tablet_manager.cc:1397] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 02:16:25.951954 5260 raft_consensus.cc:348] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0243f20817f844c3a6278a42fda5ef2d" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 40653 } } peers { permanent_uuid: "0f82b3d4504a4bd39c034262a058b0a0" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37067 } } peers { permanent_uuid: "51a1c034796345d5a7ef8336561197dd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35561 } }
I20260806 02:16:25.952231 5260 raft_consensus.cc:374] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:25.952296 5260 raft_consensus.cc:729] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f82b3d4504a4bd39c034262a058b0a0, State: Initialized, Role: FOLLOWER
I20260806 02:16:25.952744 5260 consensus_queue.cc:260] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0243f20817f844c3a6278a42fda5ef2d" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 40653 } } peers { permanent_uuid: "0f82b3d4504a4bd39c034262a058b0a0" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37067 } } peers { permanent_uuid: "51a1c034796345d5a7ef8336561197dd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35561 } }
I20260806 02:16:25.952875 5259 raft_consensus.cc:348] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0243f20817f844c3a6278a42fda5ef2d" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 40653 } } peers { permanent_uuid: "0f82b3d4504a4bd39c034262a058b0a0" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37067 } } peers { permanent_uuid: "51a1c034796345d5a7ef8336561197dd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35561 } }
I20260806 02:16:25.953132 5259 raft_consensus.cc:374] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:25.953020 5261 raft_consensus.cc:348] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0243f20817f844c3a6278a42fda5ef2d" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 40653 } } peers { permanent_uuid: "0f82b3d4504a4bd39c034262a058b0a0" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37067 } } peers { permanent_uuid: "51a1c034796345d5a7ef8336561197dd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35561 } }
I20260806 02:16:25.953198 5259 raft_consensus.cc:729] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a1c034796345d5a7ef8336561197dd, State: Initialized, Role: FOLLOWER
I20260806 02:16:25.953235 5261 raft_consensus.cc:374] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:25.953277 5261 raft_consensus.cc:729] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0243f20817f844c3a6278a42fda5ef2d, State: Initialized, Role: FOLLOWER
I20260806 02:16:25.953620 5099 heartbeater.cc:503] Master 127.18.70.126:35475 was elected leader, sending a full tablet report...
I20260806 02:16:25.953649 5261 consensus_queue.cc:260] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0243f20817f844c3a6278a42fda5ef2d" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 40653 } } peers { permanent_uuid: "0f82b3d4504a4bd39c034262a058b0a0" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37067 } } peers { permanent_uuid: "51a1c034796345d5a7ef8336561197dd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35561 } }
I20260806 02:16:25.953724 5260 ts_tablet_manager.cc:1428] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 02:16:25.954275 5240 heartbeater.cc:503] Master 127.18.70.126:35475 was elected leader, sending a full tablet report...
I20260806 02:16:25.954543 5261 ts_tablet_manager.cc:1428] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260806 02:16:25.954672 5259 consensus_queue.cc:260] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0243f20817f844c3a6278a42fda5ef2d" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 40653 } } peers { permanent_uuid: "0f82b3d4504a4bd39c034262a058b0a0" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37067 } } peers { permanent_uuid: "51a1c034796345d5a7ef8336561197dd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35561 } }
I20260806 02:16:25.955338 4951 heartbeater.cc:503] Master 127.18.70.126:35475 was elected leader, sending a full tablet report...
I20260806 02:16:25.955608 5259 ts_tablet_manager.cc:1428] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260806 02:16:25.999418 4952 tablet.cc:2369] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:16:26.086185 5266 raft_consensus.cc:482] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:16:26.086452 5266 raft_consensus.cc:504] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0243f20817f844c3a6278a42fda5ef2d" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 40653 } } peers { permanent_uuid: "0f82b3d4504a4bd39c034262a058b0a0" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37067 } } peers { permanent_uuid: "51a1c034796345d5a7ef8336561197dd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35561 } }
I20260806 02:16:26.087828 5266 leader_election.cc:288] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f82b3d4504a4bd39c034262a058b0a0 (127.18.70.66:37067), 51a1c034796345d5a7ef8336561197dd (127.18.70.65:35561)
I20260806 02:16:26.091739 5258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:26.088191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:55283 (local address 127.18.70.65:35561)
0806 02:16:26.088319 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:26.088342 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:26.088588 (+ 246us) server_negotiation.cc:407] Connection header received
0806 02:16:26.088759 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:26.088762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:26.088836 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:26.088923 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:26.089190 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:26.089676 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:26.090673 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:26.091044 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:26.091195 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:26.091246 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:26.091496 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 02:16:26.091602 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 02:16:26.091753 5257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:26.088192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:44557 (local address 127.18.70.66:37067)
0806 02:16:26.088332 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:26.088338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:26.088354 (+ 16us) server_negotiation.cc:407] Connection header received
0806 02:16:26.088536 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:26.088539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:26.088590 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:26.088718 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:26.089018 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:26.089538 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:26.090646 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:26.091027 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:26.091184 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:26.091247 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:26.091503 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 02:16:26.091607 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 02:16:26.091756 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:26.088027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:37067 (local address 127.18.70.67:44557)
0806 02:16:26.088246 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:26.088266 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:26.088366 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:26.088725 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:26.088728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:26.088747 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:26.088888 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:26.088895 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:26.089675 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:26.089682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:26.090494 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:26.090503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:26.091183 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:26.091187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:26.091351 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:26.091383 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 02:16:26.091483 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 02:16:26.091612 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":31}
I20260806 02:16:26.091763 5268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:26.088118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35561 (local address 127.18.70.67:55283)
0806 02:16:26.088497 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:26.088507 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:26.088629 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:26.088936 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:26.088939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:26.088954 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:26.089084 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:26.089089 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:26.089791 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:26.089795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:26.090555 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:26.090562 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:26.091195 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:26.091199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:26.091354 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:26.091390 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 02:16:26.091477 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 02:16:26.091598 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":60,"thread_start_us":118,"threads_started":1}
I20260806 02:16:26.092525 5043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3bc377c2f8485a9b160fe4e10034c9" candidate_uuid: "0243f20817f844c3a6278a42fda5ef2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f82b3d4504a4bd39c034262a058b0a0" is_pre_election: true
I20260806 02:16:26.092613 4902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3bc377c2f8485a9b160fe4e10034c9" candidate_uuid: "0243f20817f844c3a6278a42fda5ef2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a1c034796345d5a7ef8336561197dd" is_pre_election: true
I20260806 02:16:26.092871 5043 raft_consensus.cc:2454] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0243f20817f844c3a6278a42fda5ef2d in term 0.
I20260806 02:16:26.092909 4902 raft_consensus.cc:2454] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0243f20817f844c3a6278a42fda5ef2d in term 0.
I20260806 02:16:26.093397 5121 leader_election.cc:302] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0243f20817f844c3a6278a42fda5ef2d, 51a1c034796345d5a7ef8336561197dd; no voters:
I20260806 02:16:26.093700 5266 raft_consensus.cc:2781] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:16:26.093801 5266 raft_consensus.cc:482] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:16:26.093852 5266 raft_consensus.cc:3037] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:26.094985 5266 raft_consensus.cc:504] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0243f20817f844c3a6278a42fda5ef2d" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 40653 } } peers { permanent_uuid: "0f82b3d4504a4bd39c034262a058b0a0" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37067 } } peers { permanent_uuid: "51a1c034796345d5a7ef8336561197dd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35561 } }
I20260806 02:16:26.095353 5266 leader_election.cc:288] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [CANDIDATE]: Term 1 election: Requested vote from peers 0f82b3d4504a4bd39c034262a058b0a0 (127.18.70.66:37067), 51a1c034796345d5a7ef8336561197dd (127.18.70.65:35561)
I20260806 02:16:26.095724 5043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3bc377c2f8485a9b160fe4e10034c9" candidate_uuid: "0243f20817f844c3a6278a42fda5ef2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f82b3d4504a4bd39c034262a058b0a0"
I20260806 02:16:26.095791 4902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3bc377c2f8485a9b160fe4e10034c9" candidate_uuid: "0243f20817f844c3a6278a42fda5ef2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51a1c034796345d5a7ef8336561197dd"
I20260806 02:16:26.095858 5043 raft_consensus.cc:3037] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:26.095934 4902 raft_consensus.cc:3037] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:26.096933 4902 raft_consensus.cc:2454] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0243f20817f844c3a6278a42fda5ef2d in term 1.
I20260806 02:16:26.096949 5043 raft_consensus.cc:2454] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0243f20817f844c3a6278a42fda5ef2d in term 1.
I20260806 02:16:26.097223 5121 leader_election.cc:302] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0243f20817f844c3a6278a42fda5ef2d, 51a1c034796345d5a7ef8336561197dd; no voters:
I20260806 02:16:26.097502 5266 raft_consensus.cc:2781] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:26.097761 5266 raft_consensus.cc:686] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [term 1 LEADER]: Becoming Leader. State: Replica: 0243f20817f844c3a6278a42fda5ef2d, State: Running, Role: LEADER
I20260806 02:16:26.098096 5266 consensus_queue.cc:237] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0243f20817f844c3a6278a42fda5ef2d" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 40653 } } peers { permanent_uuid: "0f82b3d4504a4bd39c034262a058b0a0" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37067 } } peers { permanent_uuid: "51a1c034796345d5a7ef8336561197dd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35561 } }
I20260806 02:16:26.101621 4748 catalog_manager.cc:5495] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0243f20817f844c3a6278a42fda5ef2d (127.18.70.67). New cstate: current_term: 1 leader_uuid: "0243f20817f844c3a6278a42fda5ef2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0243f20817f844c3a6278a42fda5ef2d" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 40653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f82b3d4504a4bd39c034262a058b0a0" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a1c034796345d5a7ef8336561197dd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35561 } health_report { overall_health: UNKNOWN } } }
I20260806 02:16:26.119138 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:26.115605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51828 (local address 127.18.70.67:40653)
0806 02:16:26.115733 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:26.115739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:26.115754 (+ 15us) server_negotiation.cc:407] Connection header received
0806 02:16:26.115951 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:26.115954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:26.116005 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:26.116113 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:16:26.116351 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:26.116846 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:26.117684 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:26.117904 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:26.118011 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:26.118315 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:16:26.118737 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:16:26.118871 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:26.118933 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 02:16:26.118983 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 02:16:26.125190 5043 raft_consensus.cc:1261] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0 [term 1 FOLLOWER]: Refusing update from remote peer 0243f20817f844c3a6278a42fda5ef2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:16:26.125191 4902 raft_consensus.cc:1261] T 3e3bc377c2f8485a9b160fe4e10034c9 P 51a1c034796345d5a7ef8336561197dd [term 1 FOLLOWER]: Refusing update from remote peer 0243f20817f844c3a6278a42fda5ef2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:16:26.125919 5266 consensus_queue.cc:1035] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "51a1c034796345d5a7ef8336561197dd" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:16:26.126089 5269 consensus_queue.cc:1035] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f82b3d4504a4bd39c034262a058b0a0" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 37067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:16:26.132797 5272 mvcc.cc:205] Tried to move back new op lower bound from 7315384672760590336 to 7315384672660975616. Current Snapshot: MvccSnapshot[applied={T|T < 7315384672760590336}]
I20260806 02:16:26.134382 5274 mvcc.cc:205] Tried to move back new op lower bound from 7315384672760590336 to 7315384672660975616. Current Snapshot: MvccSnapshot[applied={T|T < 7315384672760590336}]
I20260806 02:16:26.136864 5273 mvcc.cc:205] Tried to move back new op lower bound from 7315384672760590336 to 7315384672660975616. Current Snapshot: MvccSnapshot[applied={T|T < 7315384672760590336}]
I20260806 02:16:26.143321 4748 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:16:26.143450 4748 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:16:26.145787 4748 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c03fda0efe2849fc980df3cda7a99d8f: Sending DeleteTablet for 3 replicas of tablet 3e3bc377c2f8485a9b160fe4e10034c9
I20260806 02:16:26.146646 4881 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3bc377c2f8485a9b160fe4e10034c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:26 UTC) from {username='slave'} at 127.0.0.1:41470
I20260806 02:16:26.146663 5024 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3bc377c2f8485a9b160fe4e10034c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:26 UTC) from {username='slave'} at 127.0.0.1:49446
I20260806 02:16:26.146718 5168 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3bc377c2f8485a9b160fe4e10034c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:26 UTC) from {username='slave'} at 127.0.0.1:51814
I20260806 02:16:26.147145 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 4814
I20260806 02:16:26.147328 5283 tablet_replica.cc:330] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0: stopping tablet replica
I20260806 02:16:26.147784 5283 raft_consensus.cc:2229] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:16:26.148025 5284 tablet_replica.cc:330] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d: stopping tablet replica
I20260806 02:16:26.148164 5283 raft_consensus.cc:2258] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:16:26.148524 5284 raft_consensus.cc:2229] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:16:26.148957 5284 raft_consensus.cc:2258] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:16:26.149740 5283 ts_tablet_manager.cc:1905] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:16:26.150388 5284 ts_tablet_manager.cc:1905] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:16:26.153239 5283 ts_tablet_manager.cc:1918] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:16:26.153345 5283 log.cc:1196] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/3e3bc377c2f8485a9b160fe4e10034c9
I20260806 02:16:26.153684 5283 ts_tablet_manager.cc:1939] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0f82b3d4504a4bd39c034262a058b0a0: Deleting consensus metadata
I20260806 02:16:26.153836 5284 ts_tablet_manager.cc:1918] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:16:26.153995 5284 log.cc:1196] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/3e3bc377c2f8485a9b160fe4e10034c9
I20260806 02:16:26.154415 5284 ts_tablet_manager.cc:1939] T 3e3bc377c2f8485a9b160fe4e10034c9 P 0243f20817f844c3a6278a42fda5ef2d: Deleting consensus metadata
I20260806 02:16:26.154940 4732 catalog_manager.cc:4841] TS 0f82b3d4504a4bd39c034262a058b0a0 (127.18.70.66:37067): tablet 3e3bc377c2f8485a9b160fe4e10034c9 (table test-table [id=2a5ccb46f96349d4b548f8ec0cb6902b]) successfully deleted
I20260806 02:16:26.155898 4734 catalog_manager.cc:4841] TS 0243f20817f844c3a6278a42fda5ef2d (127.18.70.67:40653): tablet 3e3bc377c2f8485a9b160fe4e10034c9 (table test-table [id=2a5ccb46f96349d4b548f8ec0cb6902b]) successfully deleted
W20260806 02:16:26.159264 4734 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:35561 (error 108)
I20260806 02:16:26.160063 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 4955
I20260806 02:16:26.160365 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:26.160063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35561 (local address 127.0.0.1:41484)
0806 02:16:26.160212 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:26.160285 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:35561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 02:16:26.161814 4734 catalog_manager.cc:4584] TS 51a1c034796345d5a7ef8336561197dd (127.18.70.65:35561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e3bc377c2f8485a9b160fe4e10034c9: Network error: Client connection negotiation failed: client connection to 127.18.70.65:35561: connect: Connection refused (error 111)
I20260806 02:16:26.169922 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 5102
I20260806 02:16:26.177043 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 4716
2026-08-06T02:16:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1109 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:26 dist-test-slave-6k22 krb5kdc[5288](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:26 dist-test-slave-6k22 krb5kdc[5288](info): set up 2 sockets
Aug 06 02:16:26 dist-test-slave-6k22 krb5kdc[5288](info): commencing operation
krb5kdc: starting...
W20260806 02:16:27.241029 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:27 dist-test-slave-6k22 krb5kdc[5288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:16:27.363289 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:45825
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:45825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:27.473378 5307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:27.473662 5307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:27.473738 5307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:27.477573 5307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:27.477654 5307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:27.477689 5307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:27.477725 5307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:27.477761 5307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:27.482986 5307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:45825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:45825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:27.484035 5307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:27.485033 5307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:27.493877 5312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:27.493927 5313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:27.493928 5315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:27.494192 5307 server_base.cc:1034] running on GCE node
I20260806 02:16:27.494680 5307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:27.495766 5307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:27.496930 5307 hybrid_clock.cc:648] HybridClock initialized: now 1785982587496898 us; error 40 us; skew 500 ppm
Aug 06 02:16:27 dist-test-slave-6k22 krb5kdc[5288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982587, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:27.504153 5307 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:16:27.505018 5307 webserver.cc:460] Webserver started at http://127.18.70.126:43827/ using document root <none> and password file <none>
I20260806 02:16:27.505584 5307 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:27.505645 5307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:27.505872 5307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:27.507977 5307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "d1d60e1898474ff1861bafdaee9b0435"
format_stamp: "Formatted at 2026-08-06 02:16:27 on dist-test-slave-6k22"
server_key: "2319f1b57901f9662b5934ef51026888"
server_key_iv: "fd74eb5018214cdd668afcf43066b272"
server_key_version: "encryptionkey@0"
I20260806 02:16:27.508533 5307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "d1d60e1898474ff1861bafdaee9b0435"
format_stamp: "Formatted at 2026-08-06 02:16:27 on dist-test-slave-6k22"
server_key: "2319f1b57901f9662b5934ef51026888"
server_key_iv: "fd74eb5018214cdd668afcf43066b272"
server_key_version: "encryptionkey@0"
I20260806 02:16:27.512316 5307 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 02:16:27.514671 5323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:27.515842 5307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 02:16:27.515990 5307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "d1d60e1898474ff1861bafdaee9b0435"
format_stamp: "Formatted at 2026-08-06 02:16:27 on dist-test-slave-6k22"
server_key: "2319f1b57901f9662b5934ef51026888"
server_key_iv: "fd74eb5018214cdd668afcf43066b272"
server_key_version: "encryptionkey@0"
I20260806 02:16:27.516098 5307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:27.526481 5307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:27.529774 5307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:27.529974 5307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:27.537165 5307 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:45825
I20260806 02:16:27.537192 5379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:45825 every 8 connection(s)
I20260806 02:16:27.538092 5307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:27.539915 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 5307
I20260806 02:16:27.540066 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:27.540373 18713 external_mini_cluster.cc:1428] Setting key 0933db9f532bd34c01731ec57b2842a2
I20260806 02:16:27.541157 5380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:27.547947 5380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435: Bootstrap starting.
Aug 06 02:16:27 dist-test-slave-6k22 krb5kdc[5288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:16:27.550786 5380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:27.551517 5380 log.cc:824] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:27.553773 5380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435: No bootstrap required, opened a new log
I20260806 02:16:27.555481 5384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:27.541667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.18.70.126:45825)
0806 02:16:27.542078 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:27.542097 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:27.542140 (+ 43us) server_negotiation.cc:407] Connection header received
0806 02:16:27.543001 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:27.543024 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:27.543433 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:27.543830 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:27.544492 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:27.545938 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:27.546723 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:27.547391 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:27.547574 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:27.550298 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:27.550304 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:27.550320 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:27.550350 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:27.552505 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:27.553001 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:27.553006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:27.553011 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:27.553134 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:27.553458 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:27.553460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:27.553462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:27.553936 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:27.554116 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:27.554405 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 02:16:27.554724 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":124,"threads_started":1}
I20260806 02:16:27.557377 5380 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d60e1898474ff1861bafdaee9b0435" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45825 } }
I20260806 02:16:27.557611 5380 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:27.557669 5380 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d60e1898474ff1861bafdaee9b0435, State: Initialized, Role: FOLLOWER
I20260806 02:16:27.558069 5380 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d60e1898474ff1861bafdaee9b0435" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45825 } }
I20260806 02:16:27.558228 5380 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:27.558310 5380 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:27.558467 5380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:27.559345 5380 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d60e1898474ff1861bafdaee9b0435" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45825 } }
I20260806 02:16:27.559736 5380 leader_election.cc:302] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1d60e1898474ff1861bafdaee9b0435; no voters:
I20260806 02:16:27.560045 5380 leader_election.cc:288] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:27.560236 5386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:27.560529 5386 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [term 1 LEADER]: Becoming Leader. State: Replica: d1d60e1898474ff1861bafdaee9b0435, State: Running, Role: LEADER
I20260806 02:16:27.560834 5386 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d60e1898474ff1861bafdaee9b0435" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45825 } }
I20260806 02:16:27.561305 5380 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:27.562516 5387 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1d60e1898474ff1861bafdaee9b0435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d60e1898474ff1861bafdaee9b0435" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45825 } } }
I20260806 02:16:27.562537 5388 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1d60e1898474ff1861bafdaee9b0435. Latest consensus state: current_term: 1 leader_uuid: "d1d60e1898474ff1861bafdaee9b0435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d60e1898474ff1861bafdaee9b0435" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45825 } } }
I20260806 02:16:27.562634 5387 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:27.562644 5388 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:27.563048 5395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:27.566273 5395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:27.571501 5395 catalog_manager.cc:1347] Generated new cluster ID: 15ba4e42cf5a47b6bf23ab9c1b6c3bc8
I20260806 02:16:27.571580 5395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:27.579358 5395 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:27.580261 5395 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:27.586639 5395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435: Generated new TSK 0
I20260806 02:16:27.587217 5395 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:16:27.618947 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45825
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:27.742043 5410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:27.742274 5410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:27.742314 5410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:27.746459 5410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:27.746548 5410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:27.746676 5410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:16:27.752126 5410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:27.753130 5410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:27.754370 5410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:27.764444 5418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:27.764444 5415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:27.764518 5416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:27.764724 5410 server_base.cc:1034] running on GCE node
I20260806 02:16:27.765169 5410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:27.765898 5410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:27.767071 5410 hybrid_clock.cc:648] HybridClock initialized: now 1785982587767048 us; error 30 us; skew 500 ppm
Aug 06 02:16:27 dist-test-slave-6k22 krb5kdc[5288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982587, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:27.770202 5410 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:16:27.771113 5410 webserver.cc:460] Webserver started at http://127.18.70.65:43625/ using document root <none> and password file <none>
I20260806 02:16:27.771708 5410 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:27.771781 5410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:27.771958 5410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:27.773937 5410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "388ae4f407074652bcec257287034f88"
format_stamp: "Formatted at 2026-08-06 02:16:27 on dist-test-slave-6k22"
server_key: "c99a22e0b2b41816177c73ccdcc569b2"
server_key_iv: "eddee07eebe179aeac2d1eef4b126602"
server_key_version: "encryptionkey@0"
I20260806 02:16:27.774473 5410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "388ae4f407074652bcec257287034f88"
format_stamp: "Formatted at 2026-08-06 02:16:27 on dist-test-slave-6k22"
server_key: "c99a22e0b2b41816177c73ccdcc569b2"
server_key_iv: "eddee07eebe179aeac2d1eef4b126602"
server_key_version: "encryptionkey@0"
I20260806 02:16:27.778283 5410 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 02:16:27.780920 5426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:27.782336 5410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:16:27.782481 5410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "388ae4f407074652bcec257287034f88"
format_stamp: "Formatted at 2026-08-06 02:16:27 on dist-test-slave-6k22"
server_key: "c99a22e0b2b41816177c73ccdcc569b2"
server_key_iv: "eddee07eebe179aeac2d1eef4b126602"
server_key_version: "encryptionkey@0"
I20260806 02:16:27.782593 5410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:27.801755 5410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:27.804709 5410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:27.804901 5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:27.805469 5410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:27.806511 5410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:27.806591 5410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:27.806645 5410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:27.806676 5410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:27.815369 5410 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:35645
I20260806 02:16:27.815399 5549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:35645 every 8 connection(s)
I20260806 02:16:27.816232 5410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
Aug 06 02:16:27 dist-test-slave-6k22 krb5kdc[5288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982587, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:16:27.825925 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 5410
I20260806 02:16:27.826066 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:16:27.826423 18713 external_mini_cluster.cc:1428] Setting key e3b008ca989e323c3d5659e6f6ef4398
I20260806 02:16:27.829792 5384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:27.818033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:52629 (local address 127.18.70.126:45825)
0806 02:16:27.818205 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:27.818210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:27.819014 (+ 804us) server_negotiation.cc:407] Connection header received
0806 02:16:27.820496 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:27.820498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:27.820544 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:27.820617 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:27.821772 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:27.822298 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:27.823391 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:27.823683 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:27.823760 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:27.826119 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:27.826125 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:27.826129 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:27.826162 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:27.827770 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:27.828296 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:27.828300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:27.828302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:27.828383 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:27.828737 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:27.828741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:27.828743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:27.828926 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:27.829035 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:27.829505 (+ 470us) server_negotiation.cc:299] Negotiation successful
0806 02:16:27.829622 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 02:16:27.830716 5552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:27.818408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45825 (local address 127.18.70.65:52629)
0806 02:16:27.818855 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:27.818915 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:27.820286 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:27.820824 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:27.820831 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:27.821176 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:27.821626 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:27.821639 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:27.822412 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:27.822414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:27.823282 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:27.823291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:27.823811 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:27.823814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:27.823946 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:27.824423 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:16:27.824438 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:27.825910 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:27.827930 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:27.827935 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:27.827946 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:27.828187 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:27.828481 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:27.828484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:27.828486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:27.828618 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:27.829036 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:27.829044 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:27.829270 (+ 226us) client_negotiation.cc:769] Sending connection context
0806 02:16:27.829450 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 02:16:27.829760 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":115,"threads_started":1}
I20260806 02:16:27.832036 5550 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45825
I20260806 02:16:27.832291 5550 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:27.832849 5550 heartbeater.cc:511] Master 127.18.70.126:45825 requested a full tablet report, sending...
I20260806 02:16:27.834457 5342 ts_manager.cc:194] Registered new tserver with Master: 388ae4f407074652bcec257287034f88 (127.18.70.65:35645)
I20260806 02:16:27.836077 5342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:52629
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:16:27.856118 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45825
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:27.967648 5555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:27.967891 5555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:27.967969 5555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:27.972070 5555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:27.972121 5555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:27.972222 5555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:16:27.977299 5555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:27.978343 5555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:27.979425 5555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:27.989157 5563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:27.989187 5561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:27.989177 5560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:27.989615 5555 server_base.cc:1034] running on GCE node
I20260806 02:16:27.989980 5555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:27.990589 5555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:27.991744 5555 hybrid_clock.cc:648] HybridClock initialized: now 1785982587991730 us; error 36 us; skew 500 ppm
Aug 06 02:16:27 dist-test-slave-6k22 krb5kdc[5288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982587, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:27.994427 5555 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:16:27.995217 5555 webserver.cc:460] Webserver started at http://127.18.70.66:36215/ using document root <none> and password file <none>
I20260806 02:16:27.995826 5555 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:27.995913 5555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:27.996138 5555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:27.998064 5555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "af74ecd82e4c4402900b8df256293139"
format_stamp: "Formatted at 2026-08-06 02:16:27 on dist-test-slave-6k22"
server_key: "1190b56b7fed8e98669cf37dac7ecf42"
server_key_iv: "97f41ee982b32fb95959e3390580a428"
server_key_version: "encryptionkey@0"
I20260806 02:16:27.998739 5555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "af74ecd82e4c4402900b8df256293139"
format_stamp: "Formatted at 2026-08-06 02:16:27 on dist-test-slave-6k22"
server_key: "1190b56b7fed8e98669cf37dac7ecf42"
server_key_iv: "97f41ee982b32fb95959e3390580a428"
server_key_version: "encryptionkey@0"
I20260806 02:16:28.002521 5555 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 02:16:28.005016 5570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:28.006192 5555 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:16:28.006335 5555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "af74ecd82e4c4402900b8df256293139"
format_stamp: "Formatted at 2026-08-06 02:16:27 on dist-test-slave-6k22"
server_key: "1190b56b7fed8e98669cf37dac7ecf42"
server_key_iv: "97f41ee982b32fb95959e3390580a428"
server_key_version: "encryptionkey@0"
I20260806 02:16:28.006443 5555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:28.025765 5555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:28.028539 5555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:28.028735 5555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:28.029347 5555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:28.030169 5555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:28.030236 5555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:28.030308 5555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:28.030362 5555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:28.039300 5555 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:40845
I20260806 02:16:28.039331 5695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:40845 every 8 connection(s)
I20260806 02:16:28.040226 5555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:16:28.042248 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 5555
I20260806 02:16:28.042393 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:16:28.042752 18713 external_mini_cluster.cc:1428] Setting key 3bba9f4155c7a4b24cb6d9578654e568
Aug 06 02:16:28 dist-test-slave-6k22 krb5kdc[5288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982587, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
I20260806 02:16:28.055639 5384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.041981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:44829 (local address 127.18.70.126:45825)
0806 02:16:28.042109 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:28.042112 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:28.043162 (+ 1050us) server_negotiation.cc:407] Connection header received
0806 02:16:28.044768 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:28.044774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:28.044838 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:28.044928 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:28.046041 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.046711 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.048441 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.048644 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.048723 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.051660 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:28.051665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:28.051671 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:28.051707 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:28.053469 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:28.053994 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:28.053998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:28.054001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:28.054103 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:28.054467 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:28.054470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:28.054471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:28.054644 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:28.054720 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:28.055368 (+ 648us) server_negotiation.cc:299] Negotiation successful
0806 02:16:28.055467 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 02:16:28.056442 5698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.042457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45825 (local address 127.18.70.66:44829)
0806 02:16:28.043002 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:28.043043 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:28.044538 (+ 1495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:28.045088 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:28.045094 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:28.045428 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:28.045896 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.045910 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.046838 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.046843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.048306 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.048321 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.048812 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.048819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.049055 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.049774 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:16:28.049797 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:28.051469 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:28.053591 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:28.053595 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:28.053607 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:28.053869 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:28.054214 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:28.054216 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:28.054217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:28.054354 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:28.054758 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:28.054764 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:28.055099 (+ 335us) client_negotiation.cc:769] Sending connection context
0806 02:16:28.055272 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 02:16:28.055528 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":152,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
I20260806 02:16:28.057684 5696 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45825
I20260806 02:16:28.057942 5696 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:28.058441 5696 heartbeater.cc:511] Master 127.18.70.126:45825 requested a full tablet report, sending...
I20260806 02:16:28.059506 5342 ts_manager.cc:194] Registered new tserver with Master: af74ecd82e4c4402900b8df256293139 (127.18.70.66:40845)
I20260806 02:16:28.060417 5342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:44829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:16:28.072783 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45825
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:43169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:28.186890 5701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:28.187161 5701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:28.187232 5701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:28.191143 5701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:28.191195 5701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:28.191287 5701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:16:28.196275 5701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:43169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:28.197225 5701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:28.198171 5701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:28.207477 5709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:28.207459 5707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:28.207449 5706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:28.207835 5701 server_base.cc:1034] running on GCE node
I20260806 02:16:28.208318 5701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:28.208890 5701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:28.210047 5701 hybrid_clock.cc:648] HybridClock initialized: now 1785982588209989 us; error 68 us; skew 500 ppm
Aug 06 02:16:28 dist-test-slave-6k22 krb5kdc[5288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982588, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:28.212775 5701 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:16:28.213572 5701 webserver.cc:460] Webserver started at http://127.18.70.67:36019/ using document root <none> and password file <none>
I20260806 02:16:28.214068 5701 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:28.214123 5701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:28.214303 5701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:28.216326 5701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "06aefa4087914a9f94d57c8b3ee95d8e"
format_stamp: "Formatted at 2026-08-06 02:16:28 on dist-test-slave-6k22"
server_key: "0b1e1331e8d9fc2c6ba8a9736cfe6981"
server_key_iv: "0d024ebf40829319966275a6ab76e1c4"
server_key_version: "encryptionkey@0"
I20260806 02:16:28.216893 5701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "06aefa4087914a9f94d57c8b3ee95d8e"
format_stamp: "Formatted at 2026-08-06 02:16:28 on dist-test-slave-6k22"
server_key: "0b1e1331e8d9fc2c6ba8a9736cfe6981"
server_key_iv: "0d024ebf40829319966275a6ab76e1c4"
server_key_version: "encryptionkey@0"
I20260806 02:16:28.220585 5701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 02:16:28.222805 5716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:28.224047 5701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 02:16:28.224138 5701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "06aefa4087914a9f94d57c8b3ee95d8e"
format_stamp: "Formatted at 2026-08-06 02:16:28 on dist-test-slave-6k22"
server_key: "0b1e1331e8d9fc2c6ba8a9736cfe6981"
server_key_iv: "0d024ebf40829319966275a6ab76e1c4"
server_key_version: "encryptionkey@0"
I20260806 02:16:28.224242 5701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:28.241070 5701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:28.244326 5701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:28.244469 5701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:28.244959 5701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:28.245688 5701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:28.245731 5701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:28.245767 5701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:28.245786 5701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:28.253719 5701 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:38915
I20260806 02:16:28.253793 5846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:38915 every 8 connection(s)
I20260806 02:16:28.254593 5701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:16:28.259289 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 5701
I20260806 02:16:28.259433 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:16:28.259771 18713 external_mini_cluster.cc:1428] Setting key 2134391bc2f3d6064182835946d443ab
Aug 06 02:16:28 dist-test-slave-6k22 krb5kdc[5288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982588, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:16:28.268105 5384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.256349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:47923 (local address 127.18.70.126:45825)
0806 02:16:28.256493 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:28.256498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:28.257337 (+ 839us) server_negotiation.cc:407] Connection header received
0806 02:16:28.258634 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:28.258636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:28.258683 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:28.258758 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:28.260070 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.260870 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.262500 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.262745 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.262852 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.265220 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:28.265223 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:28.265224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:28.265247 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:28.266435 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:28.266868 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:28.266870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:28.266872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:28.266941 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:28.267227 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:28.267229 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:28.267230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:28.267384 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:28.267456 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:28.267897 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 02:16:28.267991 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:16:28.268919 5849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.256744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45825 (local address 127.18.70.67:47923)
0806 02:16:28.257172 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:28.257229 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:28.258440 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:28.258921 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:28.258928 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:28.259263 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:28.259927 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.259943 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.260999 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.261002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.262294 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.262304 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.262901 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.262905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.263085 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.263713 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:16:28.263729 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:28.265064 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:28.266535 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:28.266539 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:28.266551 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:28.266784 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:28.267026 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:28.267028 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:28.267029 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:28.267149 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:28.267466 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:28.267471 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:28.267699 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 02:16:28.267851 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 02:16:28.268079 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":110,"threads_started":1}
I20260806 02:16:28.269958 5847 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45825
I20260806 02:16:28.270179 5847 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:28.270596 5847 heartbeater.cc:511] Master 127.18.70.126:45825 requested a full tablet report, sending...
I20260806 02:16:28.271490 5342 ts_manager.cc:194] Registered new tserver with Master: 06aefa4087914a9f94d57c8b3ee95d8e (127.18.70.67:38915)
I20260806 02:16:28.272054 5342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:47923
I20260806 02:16:28.273540 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:16:28.282915 5384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.275399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.18.70.126:45825)
0806 02:16:28.275559 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:28.275563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:28.275780 (+ 217us) server_negotiation.cc:407] Connection header received
0806 02:16:28.275936 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:28.275939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:28.275982 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:28.276060 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:28.276735 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.277231 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.278064 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.278282 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.278406 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.280015 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:28.280021 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:28.280028 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:28.280050 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:28.281342 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:28.281809 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:28.281812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:28.281813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:28.281881 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:28.282143 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:28.282145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:28.282146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:28.282334 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:28.282447 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:28.282736 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 02:16:28.282808 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 02:16:28.285632 5342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:16:28.288770 5342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:16:28.303228 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.297813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50792 (local address 127.18.70.66:40845)
0806 02:16:28.298967 (+ 1154us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:28.298981 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:28.299004 (+ 23us) server_negotiation.cc:407] Connection header received
0806 02:16:28.299179 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:28.299199 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:28.299384 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:28.299534 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:28.299763 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.300568 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.301934 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.302498 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.302629 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.302781 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:28.302958 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 02:16:28.303061 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1043,"thread_start_us":130,"threads_started":1}
I20260806 02:16:28.303247 5858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.296893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:38915 (local address 127.0.0.1:50854)
0806 02:16:28.297237 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:28.297286 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:28.297425 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:28.298677 (+ 1252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:28.298680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:28.298700 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:28.298889 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.298899 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.300406 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.300410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.301665 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.301675 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.302773 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.302777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.302864 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.302887 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 02:16:28.302940 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 02:16:28.303061 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":24,"thread_start_us":69,"threads_started":1}
I20260806 02:16:28.303584 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.297672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33302 (local address 127.18.70.65:35645)
0806 02:16:28.298127 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:28.298145 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:28.298164 (+ 19us) server_negotiation.cc:407] Connection header received
0806 02:16:28.298253 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:28.298258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:28.298494 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:28.298657 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:28.299134 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.300246 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.301889 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.302628 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.302755 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.302894 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:28.303300 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 02:16:28.303399 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":147,"threads_started":1}
I20260806 02:16:28.303663 5860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.297112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35645 (local address 127.0.0.1:33302)
0806 02:16:28.297803 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:28.297817 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:28.297911 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:28.298645 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:28.298652 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:28.298673 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:28.299017 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.299023 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.300443 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.300446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.301767 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.301778 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.302954 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.302958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.303039 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.303060 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 02:16:28.303128 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 02:16:28.303187 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":81,"thread_start_us":123,"threads_started":1}
I20260806 02:16:28.303756 5859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.297050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50854 (local address 127.18.70.67:38915)
0806 02:16:28.298158 (+ 1108us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:28.298170 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:28.298194 (+ 24us) server_negotiation.cc:407] Connection header received
0806 02:16:28.298276 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:28.298287 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:28.298515 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:28.298878 (+ 363us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:28.299019 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.300246 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.301797 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.302627 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.303318 (+ 691us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.303446 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:28.303496 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 02:16:28.303606 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1003,"thread_start_us":108,"threads_started":1}
I20260806 02:16:28.303229 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.297588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:40845 (local address 127.0.0.1:50792)
0806 02:16:28.297919 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:28.297933 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:28.298023 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:28.299514 (+ 1491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:28.299517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:28.299535 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:28.299654 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.299660 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.300686 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.300689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.301788 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.301794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.302629 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.302632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.302739 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.302776 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 02:16:28.302926 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 02:16:28.303074 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":25,"thread_start_us":111,"threads_started":1}
I20260806 02:16:28.306468 5768 tablet_service.cc:1467] Processing CreateTablet for tablet f6d59266273f4eba85ee250ceb82c628 (DEFAULT_TABLE table=test-table [id=6bc880cc86e74b33b16fb07ac6f77624]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:28.306546 5624 tablet_service.cc:1467] Processing CreateTablet for tablet f6d59266273f4eba85ee250ceb82c628 (DEFAULT_TABLE table=test-table [id=6bc880cc86e74b33b16fb07ac6f77624]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:28.306840 5477 tablet_service.cc:1467] Processing CreateTablet for tablet f6d59266273f4eba85ee250ceb82c628 (DEFAULT_TABLE table=test-table [id=6bc880cc86e74b33b16fb07ac6f77624]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:28.307512 5768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6d59266273f4eba85ee250ceb82c628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:28.307596 5624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6d59266273f4eba85ee250ceb82c628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:28.307722 5477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6d59266273f4eba85ee250ceb82c628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:28.313738 5864 tablet_bootstrap.cc:492] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e: Bootstrap starting.
I20260806 02:16:28.314962 5866 tablet_bootstrap.cc:492] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139: Bootstrap starting.
I20260806 02:16:28.315331 5867 tablet_bootstrap.cc:492] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88: Bootstrap starting.
I20260806 02:16:28.317269 5864 tablet_bootstrap.cc:654] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:28.317296 5867 tablet_bootstrap.cc:654] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:28.317786 5866 tablet_bootstrap.cc:654] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:28.318481 5867 log.cc:824] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:28.318629 5866 log.cc:824] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:28.318629 5864 log.cc:824] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:28.320618 5867 tablet_bootstrap.cc:492] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88: No bootstrap required, opened a new log
I20260806 02:16:28.320703 5866 tablet_bootstrap.cc:492] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139: No bootstrap required, opened a new log
I20260806 02:16:28.320822 5867 ts_tablet_manager.cc:1397] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 02:16:28.320904 5866 ts_tablet_manager.cc:1397] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:16:28.321117 5864 tablet_bootstrap.cc:492] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e: No bootstrap required, opened a new log
I20260806 02:16:28.321287 5864 ts_tablet_manager.cc:1397] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 02:16:28.323822 5867 raft_consensus.cc:348] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 38915 } } peers { permanent_uuid: "af74ecd82e4c4402900b8df256293139" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 40845 } } peers { permanent_uuid: "388ae4f407074652bcec257287034f88" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35645 } }
I20260806 02:16:28.324049 5867 raft_consensus.cc:374] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:28.324103 5867 raft_consensus.cc:729] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 388ae4f407074652bcec257287034f88, State: Initialized, Role: FOLLOWER
I20260806 02:16:28.324548 5867 consensus_queue.cc:260] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 38915 } } peers { permanent_uuid: "af74ecd82e4c4402900b8df256293139" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 40845 } } peers { permanent_uuid: "388ae4f407074652bcec257287034f88" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35645 } }
I20260806 02:16:28.324760 5866 raft_consensus.cc:348] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 38915 } } peers { permanent_uuid: "af74ecd82e4c4402900b8df256293139" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 40845 } } peers { permanent_uuid: "388ae4f407074652bcec257287034f88" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35645 } }
I20260806 02:16:28.325058 5866 raft_consensus.cc:374] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:28.324900 5864 raft_consensus.cc:348] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 38915 } } peers { permanent_uuid: "af74ecd82e4c4402900b8df256293139" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 40845 } } peers { permanent_uuid: "388ae4f407074652bcec257287034f88" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35645 } }
I20260806 02:16:28.325119 5866 raft_consensus.cc:729] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af74ecd82e4c4402900b8df256293139, State: Initialized, Role: FOLLOWER
I20260806 02:16:28.325155 5864 raft_consensus.cc:374] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:28.325212 5864 raft_consensus.cc:729] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06aefa4087914a9f94d57c8b3ee95d8e, State: Initialized, Role: FOLLOWER
I20260806 02:16:28.325414 5550 heartbeater.cc:503] Master 127.18.70.126:45825 was elected leader, sending a full tablet report...
I20260806 02:16:28.325587 5867 ts_tablet_manager.cc:1428] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 02:16:28.325608 5864 consensus_queue.cc:260] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 38915 } } peers { permanent_uuid: "af74ecd82e4c4402900b8df256293139" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 40845 } } peers { permanent_uuid: "388ae4f407074652bcec257287034f88" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35645 } }
I20260806 02:16:28.325608 5866 consensus_queue.cc:260] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 38915 } } peers { permanent_uuid: "af74ecd82e4c4402900b8df256293139" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 40845 } } peers { permanent_uuid: "388ae4f407074652bcec257287034f88" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35645 } }
I20260806 02:16:28.326243 5696 heartbeater.cc:503] Master 127.18.70.126:45825 was elected leader, sending a full tablet report...
I20260806 02:16:28.326514 5864 ts_tablet_manager.cc:1428] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 02:16:28.326514 5866 ts_tablet_manager.cc:1428] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:16:28.326808 5847 heartbeater.cc:503] Master 127.18.70.126:45825 was elected leader, sending a full tablet report...
I20260806 02:16:28.439594 5873 raft_consensus.cc:482] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:16:28.439834 5873 raft_consensus.cc:504] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 38915 } } peers { permanent_uuid: "af74ecd82e4c4402900b8df256293139" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 40845 } } peers { permanent_uuid: "388ae4f407074652bcec257287034f88" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35645 } }
I20260806 02:16:28.441068 5873 leader_election.cc:288] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af74ecd82e4c4402900b8df256293139 (127.18.70.66:40845), 388ae4f407074652bcec257287034f88 (127.18.70.65:35645)
I20260806 02:16:28.445907 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.441503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:34685 (local address 127.18.70.66:40845)
0806 02:16:28.441629 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:28.441633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:28.441648 (+ 15us) server_negotiation.cc:407] Connection header received
0806 02:16:28.441812 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:28.441815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:28.441867 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:28.441943 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:28.442295 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.443063 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.444612 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.445258 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.445379 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.445432 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:28.445699 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 02:16:28.445772 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 02:16:28.445907 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.441613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:60185 (local address 127.18.70.65:35645)
0806 02:16:28.441794 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:28.441799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:28.442083 (+ 284us) server_negotiation.cc:407] Connection header received
0806 02:16:28.442180 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:28.442183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:28.442248 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:28.442331 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:28.442595 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.443260 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.444612 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.445253 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.445379 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.445432 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:28.445717 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 02:16:28.445785 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 02:16:28.445926 5849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.441376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:40845 (local address 127.18.70.67:34685)
0806 02:16:28.441522 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:28.441541 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:28.441652 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:28.441992 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:28.441998 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:28.442025 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:28.442163 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.442171 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.443174 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.443176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.444466 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.444478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.445386 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.445390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.445542 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.445590 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 02:16:28.445663 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 02:16:28.445771 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":115}
I20260806 02:16:28.445935 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.441376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35645 (local address 127.18.70.67:60185)
0806 02:16:28.441973 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:28.441987 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:28.442084 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:28.442353 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:28.442355 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:28.442373 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:28.442493 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.442498 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.443354 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.443357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.444508 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:28.444518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.445386 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.445390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:28.445538 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.445585 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 02:16:28.445663 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 02:16:28.445771 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":167,"spinlock_wait_cycles":768,"thread_start_us":122,"threads_started":1}
I20260806 02:16:28.446628 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6d59266273f4eba85ee250ceb82c628" candidate_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af74ecd82e4c4402900b8df256293139" is_pre_election: true
I20260806 02:16:28.446931 5644 raft_consensus.cc:2454] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06aefa4087914a9f94d57c8b3ee95d8e in term 0.
I20260806 02:16:28.446911 5501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6d59266273f4eba85ee250ceb82c628" candidate_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "388ae4f407074652bcec257287034f88" is_pre_election: true
I20260806 02:16:28.447198 5501 raft_consensus.cc:2454] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06aefa4087914a9f94d57c8b3ee95d8e in term 0.
I20260806 02:16:28.447438 5717 leader_election.cc:302] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06aefa4087914a9f94d57c8b3ee95d8e, af74ecd82e4c4402900b8df256293139; no voters:
I20260806 02:16:28.447739 5873 raft_consensus.cc:2781] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:16:28.447827 5873 raft_consensus.cc:482] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:16:28.447870 5873 raft_consensus.cc:3037] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:28.448877 5873 raft_consensus.cc:504] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 38915 } } peers { permanent_uuid: "af74ecd82e4c4402900b8df256293139" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 40845 } } peers { permanent_uuid: "388ae4f407074652bcec257287034f88" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35645 } }
I20260806 02:16:28.449218 5873 leader_election.cc:288] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [CANDIDATE]: Term 1 election: Requested vote from peers af74ecd82e4c4402900b8df256293139 (127.18.70.66:40845), 388ae4f407074652bcec257287034f88 (127.18.70.65:35645)
I20260806 02:16:28.449550 5644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6d59266273f4eba85ee250ceb82c628" candidate_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af74ecd82e4c4402900b8df256293139"
I20260806 02:16:28.449561 5501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6d59266273f4eba85ee250ceb82c628" candidate_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "388ae4f407074652bcec257287034f88"
I20260806 02:16:28.449672 5501 raft_consensus.cc:3037] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:28.449672 5644 raft_consensus.cc:3037] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:28.450955 5501 raft_consensus.cc:2454] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06aefa4087914a9f94d57c8b3ee95d8e in term 1.
I20260806 02:16:28.450955 5644 raft_consensus.cc:2454] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06aefa4087914a9f94d57c8b3ee95d8e in term 1.
I20260806 02:16:28.451259 5718 leader_election.cc:302] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06aefa4087914a9f94d57c8b3ee95d8e, 388ae4f407074652bcec257287034f88; no voters:
I20260806 02:16:28.451413 5873 raft_consensus.cc:2781] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:28.451658 5873 raft_consensus.cc:686] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [term 1 LEADER]: Becoming Leader. State: Replica: 06aefa4087914a9f94d57c8b3ee95d8e, State: Running, Role: LEADER
I20260806 02:16:28.451985 5873 consensus_queue.cc:237] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 38915 } } peers { permanent_uuid: "af74ecd82e4c4402900b8df256293139" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 40845 } } peers { permanent_uuid: "388ae4f407074652bcec257287034f88" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35645 } }
I20260806 02:16:28.455556 5341 catalog_manager.cc:5495] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 06aefa4087914a9f94d57c8b3ee95d8e (127.18.70.67). New cstate: current_term: 1 leader_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06aefa4087914a9f94d57c8b3ee95d8e" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 38915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af74ecd82e4c4402900b8df256293139" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 40845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "388ae4f407074652bcec257287034f88" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35645 } health_report { overall_health: UNKNOWN } } }
I20260806 02:16:28.500840 5859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.497280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35132 (local address 127.18.70.67:38915)
0806 02:16:28.497420 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:28.497424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:28.497437 (+ 13us) server_negotiation.cc:407] Connection header received
0806 02:16:28.497510 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:28.497512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:28.497569 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:28.497660 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:16:28.497886 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.498400 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.499167 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.499376 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.499453 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.499888 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:16:28.500328 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:16:28.500439 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:28.500558 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 02:16:28.500618 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 02:16:28.505453 5848 tablet.cc:2369] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:16:28.506923 5644 raft_consensus.cc:1261] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139 [term 1 FOLLOWER]: Refusing update from remote peer 06aefa4087914a9f94d57c8b3ee95d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:16:28.506924 5501 raft_consensus.cc:1261] T f6d59266273f4eba85ee250ceb82c628 P 388ae4f407074652bcec257287034f88 [term 1 FOLLOWER]: Refusing update from remote peer 06aefa4087914a9f94d57c8b3ee95d8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:16:28.507771 5875 consensus_queue.cc:1035] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "af74ecd82e4c4402900b8df256293139" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 40845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:16:28.507978 5873 consensus_queue.cc:1035] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "388ae4f407074652bcec257287034f88" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:16:28.516292 5880 mvcc.cc:205] Tried to move back new op lower bound from 7315384682516369408 to 7315384682301952000. Current Snapshot: MvccSnapshot[applied={T|T < 7315384682516369408}]
I20260806 02:16:28.525234 5341 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:16:28.525444 5341 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:16:28.527992 5341 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1d60e1898474ff1861bafdaee9b0435: Sending DeleteTablet for 3 replicas of tablet f6d59266273f4eba85ee250ceb82c628
I20260806 02:16:28.528931 5477 tablet_service.cc:1514] Processing DeleteTablet for tablet f6d59266273f4eba85ee250ceb82c628 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:28 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:33302
I20260806 02:16:28.529088 5624 tablet_service.cc:1514] Processing DeleteTablet for tablet f6d59266273f4eba85ee250ceb82c628 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:28 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:50792
I20260806 02:16:28.529456 5768 tablet_service.cc:1514] Processing DeleteTablet for tablet f6d59266273f4eba85ee250ceb82c628 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:28 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:50854
I20260806 02:16:28.529645 5890 tablet_replica.cc:330] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139: stopping tablet replica
I20260806 02:16:28.529884 5891 tablet_replica.cc:330] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e: stopping tablet replica
I20260806 02:16:28.530026 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 5410
I20260806 02:16:28.530138 5890 raft_consensus.cc:2229] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:16:28.530184 5891 raft_consensus.cc:2229] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:16:28.530539 5890 raft_consensus.cc:2258] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:16:28.530606 5891 raft_consensus.cc:2258] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:16:28.532370 5890 ts_tablet_manager.cc:1905] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:16:28.532737 5891 ts_tablet_manager.cc:1905] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:16:28.535440 5891 ts_tablet_manager.cc:1918] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:16:28.535539 5891 log.cc:1196] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/f6d59266273f4eba85ee250ceb82c628
I20260806 02:16:28.535899 5891 ts_tablet_manager.cc:1939] T f6d59266273f4eba85ee250ceb82c628 P 06aefa4087914a9f94d57c8b3ee95d8e: Deleting consensus metadata
I20260806 02:16:28.536083 5890 ts_tablet_manager.cc:1918] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:16:28.536227 5890 log.cc:1196] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/f6d59266273f4eba85ee250ceb82c628
I20260806 02:16:28.536612 5890 ts_tablet_manager.cc:1939] T f6d59266273f4eba85ee250ceb82c628 P af74ecd82e4c4402900b8df256293139: Deleting consensus metadata
I20260806 02:16:28.537132 5324 catalog_manager.cc:4841] TS 06aefa4087914a9f94d57c8b3ee95d8e (127.18.70.67:38915): tablet f6d59266273f4eba85ee250ceb82c628 (table test-table [id=6bc880cc86e74b33b16fb07ac6f77624]) successfully deleted
I20260806 02:16:28.537873 5324 catalog_manager.cc:4841] TS af74ecd82e4c4402900b8df256293139 (127.18.70.66:40845): tablet f6d59266273f4eba85ee250ceb82c628 (table test-table [id=6bc880cc86e74b33b16fb07ac6f77624]) successfully deleted
W20260806 02:16:28.543051 5325 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:35645 (error 108)
I20260806 02:16:28.544122 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.543820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35645 (local address 127.0.0.1:56976)
0806 02:16:28.543950 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:28.544025 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:35645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 02:16:28.544250 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 5555
W20260806 02:16:28.544420 5325 catalog_manager.cc:4584] TS 388ae4f407074652bcec257287034f88 (127.18.70.65:35645): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f6d59266273f4eba85ee250ceb82c628: Network error: Client connection negotiation failed: client connection to 127.18.70.65:35645: connect: Connection refused (error 111)
I20260806 02:16:28.552763 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 5701
I20260806 02:16:28.560057 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 5307
2026-08-06T02:16:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2381 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T02:16:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:28Z Disabled control of system clock
I20260806 02:16:28.592820 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:35213
--webserver_interface=127.18.70.126
--webserver_port=0
--builtin_ntp_servers=127.18.70.84:40955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:35213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 02:16:28.704545 5896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:28.704788 5896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:28.704830 5896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:28.708704 5896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:28.708752 5896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:28.708767 5896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:28.708783 5896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:28.708798 5896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:28.713335 5896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:35213
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:35213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:28.714313 5896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:28.715680 5896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:28.723812 5905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:28.723815 5902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:28.723793 5901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:28.724200 5896 server_base.cc:1034] running on GCE node
I20260806 02:16:28.724771 5896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:28.725841 5896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:28.727032 5896 hybrid_clock.cc:648] HybridClock initialized: now 1785982588727010 us; error 43 us; skew 500 ppm
I20260806 02:16:28.729033 5896 webserver.cc:460] Webserver started at http://127.18.70.126:40903/ using document root <none> and password file <none>
I20260806 02:16:28.729619 5896 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:28.729682 5896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:28.729928 5896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:28.732026 5896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "bb95976b828445cc8496bc4a022eb706"
format_stamp: "Formatted at 2026-08-06 02:16:28 on dist-test-slave-6k22"
server_key: "e5fd8d240eacab737c2b5245e6bbfded"
server_key_iv: "350acfff773adb1e4a68c317f72244a0"
server_key_version: "encryptionkey@0"
I20260806 02:16:28.732587 5896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "bb95976b828445cc8496bc4a022eb706"
format_stamp: "Formatted at 2026-08-06 02:16:28 on dist-test-slave-6k22"
server_key: "e5fd8d240eacab737c2b5245e6bbfded"
server_key_iv: "350acfff773adb1e4a68c317f72244a0"
server_key_version: "encryptionkey@0"
I20260806 02:16:28.736265 5896 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:16:28.738760 5911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:28.740024 5896 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:28.740161 5896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "bb95976b828445cc8496bc4a022eb706"
format_stamp: "Formatted at 2026-08-06 02:16:28 on dist-test-slave-6k22"
server_key: "e5fd8d240eacab737c2b5245e6bbfded"
server_key_iv: "350acfff773adb1e4a68c317f72244a0"
server_key_version: "encryptionkey@0"
I20260806 02:16:28.740265 5896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:28.771907 5896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:28.772578 5896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:28.772794 5896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:28.779796 5896 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:35213
I20260806 02:16:28.779843 5968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:35213 every 8 connection(s)
I20260806 02:16:28.780788 5896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:28.783897 5969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:28.789037 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 5896
I20260806 02:16:28.789186 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:28.789458 5969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706: Bootstrap starting.
I20260806 02:16:28.789480 18713 external_mini_cluster.cc:1428] Setting key cfd7a70e248681595601786fcc91d7c7
I20260806 02:16:28.792106 5969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:28.793010 5969 log.cc:824] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:28.795286 5969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706: No bootstrap required, opened a new log
I20260806 02:16:28.799585 5969 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb95976b828445cc8496bc4a022eb706" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35213 } }
I20260806 02:16:28.799839 5969 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:28.799898 5969 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb95976b828445cc8496bc4a022eb706, State: Initialized, Role: FOLLOWER
I20260806 02:16:28.800369 5969 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb95976b828445cc8496bc4a022eb706" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35213 } }
I20260806 02:16:28.800515 5972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:28.791039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60974 (local address 127.18.70.126:35213)
0806 02:16:28.791787 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:28.791805 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:28.791858 (+ 53us) server_negotiation.cc:407] Connection header received
0806 02:16:28.792695 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:28.792719 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:28.793229 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:28.793570 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:28.794090 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.795491 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.796545 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:28.797265 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:28.797427 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:28.798326 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:28.798334 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:28.798356 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:28.798943 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:28.799018 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:28.799045 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:28.799148 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:28.799290 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 02:16:28.799551 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":137,"threads_started":1}
I20260806 02:16:28.800526 5969 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:28.800653 5969 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:28.800740 5969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:28.801783 5969 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb95976b828445cc8496bc4a022eb706" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35213 } }
I20260806 02:16:28.802116 5969 leader_election.cc:302] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb95976b828445cc8496bc4a022eb706; no voters:
I20260806 02:16:28.802439 5969 leader_election.cc:288] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:28.802526 5974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:28.803021 5974 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [term 1 LEADER]: Becoming Leader. State: Replica: bb95976b828445cc8496bc4a022eb706, State: Running, Role: LEADER
I20260806 02:16:28.803313 5974 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb95976b828445cc8496bc4a022eb706" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35213 } }
I20260806 02:16:28.803733 5969 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:28.805102 5975 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb95976b828445cc8496bc4a022eb706" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb95976b828445cc8496bc4a022eb706" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35213 } } }
I20260806 02:16:28.805148 5976 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb95976b828445cc8496bc4a022eb706. Latest consensus state: current_term: 1 leader_uuid: "bb95976b828445cc8496bc4a022eb706" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb95976b828445cc8496bc4a022eb706" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 35213 } } }
I20260806 02:16:28.805223 5975 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:28.805275 5976 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:28.805670 5984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:28.809307 5984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:28.814653 5984 catalog_manager.cc:1347] Generated new cluster ID: 039dc627cea742e0b33a2bc55f608fa0
I20260806 02:16:28.814714 5984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:28.831393 5984 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:28.832329 5984 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:28.848290 5984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706: Generated new TSK 0
I20260806 02:16:28.849037 5984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:28.851931 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35213
--builtin_ntp_servers=127.18.70.84:40955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 02:16:28.966424 5993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:28.966655 5993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:28.966692 5993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:28.970656 5993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:28.970708 5993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:28.970764 5993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:16:28.975275 5993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:28.976254 5993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:28.977564 5993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:28.987356 6001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:28.987365 5999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:28.987530 5993 server_base.cc:1034] running on GCE node
W20260806 02:16:28.987377 5998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:28.988132 5993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:28.988821 5993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:28.990015 5993 hybrid_clock.cc:648] HybridClock initialized: now 1785982588990000 us; error 25 us; skew 500 ppm
I20260806 02:16:28.991957 5993 webserver.cc:460] Webserver started at http://127.18.70.65:32917/ using document root <none> and password file <none>
I20260806 02:16:28.992547 5993 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:28.992645 5993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:28.992882 5993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:28.994904 5993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "6157d9b6a5f345c5aa17e45b92ece6ed"
format_stamp: "Formatted at 2026-08-06 02:16:28 on dist-test-slave-6k22"
server_key: "24bd2590f56d9b665b6e6814a55f790f"
server_key_iv: "9234048e62566329ef6d16c47ce57bf3"
server_key_version: "encryptionkey@0"
I20260806 02:16:28.995473 5993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "6157d9b6a5f345c5aa17e45b92ece6ed"
format_stamp: "Formatted at 2026-08-06 02:16:28 on dist-test-slave-6k22"
server_key: "24bd2590f56d9b665b6e6814a55f790f"
server_key_iv: "9234048e62566329ef6d16c47ce57bf3"
server_key_version: "encryptionkey@0"
I20260806 02:16:28.999157 5993 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:16:29.001657 6007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:29.002840 5993 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 02:16:29.002979 5993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "6157d9b6a5f345c5aa17e45b92ece6ed"
format_stamp: "Formatted at 2026-08-06 02:16:28 on dist-test-slave-6k22"
server_key: "24bd2590f56d9b665b6e6814a55f790f"
server_key_iv: "9234048e62566329ef6d16c47ce57bf3"
server_key_version: "encryptionkey@0"
I20260806 02:16:29.003090 5993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:29.016115 5993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:29.016716 5993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:29.016908 5993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:29.017458 5993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:29.018370 5993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:29.018440 5993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:29.018504 5993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:29.018554 5993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:29.027462 5993 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:35753
I20260806 02:16:29.027493 6128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:35753 every 8 connection(s)
I20260806 02:16:29.028326 5993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:16:29.028990 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 5993
I20260806 02:16:29.029129 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:16:29.029445 18713 external_mini_cluster.cc:1428] Setting key 0e970fbadf47b14c7144423e8f755325
I20260806 02:16:29.031682 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35213
--builtin_ntp_servers=127.18.70.84:40955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 02:16:29.039670 5972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.030272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:38639 (local address 127.18.70.126:35213)
0806 02:16:29.030396 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:29.030400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:29.031813 (+ 1413us) server_negotiation.cc:407] Connection header received
0806 02:16:29.033882 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:29.033889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:29.033952 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:29.034051 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:29.034916 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.035504 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.037397 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.037617 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.037705 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.039022 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:29.039025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:29.039027 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:29.039039 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:29.039056 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:29.039064 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:29.039117 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:29.039386 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 02:16:29.039450 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 02:16:29.040855 6131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.030787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35213 (local address 127.18.70.65:38639)
0806 02:16:29.031638 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:29.031694 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:29.033635 (+ 1941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:29.034201 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:29.034207 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:29.034288 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:29.034766 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.034779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.035682 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.035689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.037258 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.037274 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.037787 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.037793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.038033 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.038696 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:16:29.038699 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:29.038758 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:16:29.038846 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:29.039150 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:29.039163 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:29.039173 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 02:16:29.039321 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 02:16:29.039674 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":154,"threads_started":1}
I20260806 02:16:29.042210 6129 heartbeater.cc:347] Connected to a master server at 127.18.70.126:35213
I20260806 02:16:29.042474 6129 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:29.043051 6129 heartbeater.cc:511] Master 127.18.70.126:35213 requested a full tablet report, sending...
I20260806 02:16:29.044734 5930 ts_manager.cc:194] Registered new tserver with Master: 6157d9b6a5f345c5aa17e45b92ece6ed (127.18.70.65:35753)
I20260806 02:16:29.045943 5930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.70.65:38639
W20260806 02:16:29.144723 6132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:29.144985 6132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:29.145044 6132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:29.148954 6132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:29.149001 6132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:29.149055 6132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:16:29.153609 6132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:29.154564 6132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:29.155923 6132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:29.165206 6138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:29.165169 6137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:29.165138 6132 server_base.cc:1034] running on GCE node
W20260806 02:16:29.165186 6140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:29.165674 6132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:29.166297 6132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:29.167486 6132 hybrid_clock.cc:648] HybridClock initialized: now 1785982589167459 us; error 39 us; skew 500 ppm
I20260806 02:16:29.169200 6132 webserver.cc:460] Webserver started at http://127.18.70.66:45957/ using document root <none> and password file <none>
I20260806 02:16:29.169742 6132 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:29.169842 6132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:29.170066 6132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:29.172104 6132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "146f8be44d764b4fbcbf6146ea1abdd2"
format_stamp: "Formatted at 2026-08-06 02:16:29 on dist-test-slave-6k22"
server_key: "11711088768efeae09b406a0485688a4"
server_key_iv: "a637ad36223b5077438a6f782185812c"
server_key_version: "encryptionkey@0"
I20260806 02:16:29.172684 6132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "146f8be44d764b4fbcbf6146ea1abdd2"
format_stamp: "Formatted at 2026-08-06 02:16:29 on dist-test-slave-6k22"
server_key: "11711088768efeae09b406a0485688a4"
server_key_iv: "a637ad36223b5077438a6f782185812c"
server_key_version: "encryptionkey@0"
I20260806 02:16:29.176153 6132 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:16:29.178491 6146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:29.179661 6132 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:29.179783 6132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "146f8be44d764b4fbcbf6146ea1abdd2"
format_stamp: "Formatted at 2026-08-06 02:16:29 on dist-test-slave-6k22"
server_key: "11711088768efeae09b406a0485688a4"
server_key_iv: "a637ad36223b5077438a6f782185812c"
server_key_version: "encryptionkey@0"
I20260806 02:16:29.179898 6132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:29.192972 6132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:29.193529 6132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:29.193707 6132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:29.194223 6132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:29.195106 6132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:29.195174 6132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:29.195243 6132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:29.195286 6132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:29.204856 6132 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:44365
I20260806 02:16:29.204968 6268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:44365 every 8 connection(s)
I20260806 02:16:29.205705 6132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:16:29.208905 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 6132
I20260806 02:16:29.209059 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:16:29.209405 18713 external_mini_cluster.cc:1428] Setting key 3b5b3aa25ca4d484239e2c8a627ca28e
I20260806 02:16:29.211510 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35213
--builtin_ntp_servers=127.18.70.84:40955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 02:16:29.215263 5972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.207630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:47235 (local address 127.18.70.126:35213)
0806 02:16:29.207785 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:29.207791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:29.208851 (+ 1060us) server_negotiation.cc:407] Connection header received
0806 02:16:29.210450 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:29.210453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:29.210510 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:29.210590 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:29.211532 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.212186 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.213278 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.213505 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.213633 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.214740 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:29.214743 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:29.214746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:29.214765 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:29.214797 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:29.214804 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:29.214881 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:29.215071 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 02:16:29.215130 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 02:16:29.216140 6272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.208082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35213 (local address 127.18.70.66:47235)
0806 02:16:29.208705 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:29.208742 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:29.210250 (+ 1508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:29.210770 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:29.210778 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:29.210891 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:29.211390 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.211403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.212316 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.212322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.213153 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.213162 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.213674 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.213678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.213846 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.214493 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:16:29.214494 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:29.214531 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:16:29.214590 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:29.214885 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:29.214897 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:29.214904 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 02:16:29.215068 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 02:16:29.215319 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":121,"threads_started":1}
I20260806 02:16:29.217415 6270 heartbeater.cc:347] Connected to a master server at 127.18.70.126:35213
I20260806 02:16:29.217633 6270 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:29.218106 6270 heartbeater.cc:511] Master 127.18.70.126:35213 requested a full tablet report, sending...
I20260806 02:16:29.219147 5930 ts_manager.cc:194] Registered new tserver with Master: 146f8be44d764b4fbcbf6146ea1abdd2 (127.18.70.66:44365)
I20260806 02:16:29.219745 5930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.70.66:47235
W20260806 02:16:29.326992 6273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:29.327282 6273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:29.327368 6273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:29.331367 6273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:29.331440 6273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:29.331529 6273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:16:29.336144 6273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:35213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:29.337131 6273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:29.338479 6273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:29.347894 6280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:29.347879 6279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:29.347978 6282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:29.348248 6273 server_base.cc:1034] running on GCE node
I20260806 02:16:29.348788 6273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:29.349462 6273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:29.350653 6273 hybrid_clock.cc:648] HybridClock initialized: now 1785982589350627 us; error 34 us; skew 500 ppm
I20260806 02:16:29.352694 6273 webserver.cc:460] Webserver started at http://127.18.70.67:42111/ using document root <none> and password file <none>
I20260806 02:16:29.353281 6273 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:29.353349 6273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:29.353618 6273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:29.355738 6273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "6fec1eee14054a3bbb58cdb7be743a8b"
format_stamp: "Formatted at 2026-08-06 02:16:29 on dist-test-slave-6k22"
server_key: "7c2531caddfc756f4d87dc7974cb295f"
server_key_iv: "dbd5997ee28a90bfb99c32a52c6c4bd8"
server_key_version: "encryptionkey@0"
I20260806 02:16:29.356320 6273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "6fec1eee14054a3bbb58cdb7be743a8b"
format_stamp: "Formatted at 2026-08-06 02:16:29 on dist-test-slave-6k22"
server_key: "7c2531caddfc756f4d87dc7974cb295f"
server_key_iv: "dbd5997ee28a90bfb99c32a52c6c4bd8"
server_key_version: "encryptionkey@0"
I20260806 02:16:29.360059 6273 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 02:16:29.362854 6288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:29.364205 6273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:16:29.364354 6273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "6fec1eee14054a3bbb58cdb7be743a8b"
format_stamp: "Formatted at 2026-08-06 02:16:29 on dist-test-slave-6k22"
server_key: "7c2531caddfc756f4d87dc7974cb295f"
server_key_iv: "dbd5997ee28a90bfb99c32a52c6c4bd8"
server_key_version: "encryptionkey@0"
I20260806 02:16:29.364470 6273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:29.381256 6273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:29.381968 6273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:29.382174 6273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:29.382920 6273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:29.384029 6273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:29.384095 6273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:29.384158 6273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:29.384207 6273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:29.393285 6273 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:45527
I20260806 02:16:29.393337 6411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:45527 every 8 connection(s)
I20260806 02:16:29.394219 6273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:16:29.399219 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 6273
I20260806 02:16:29.399348 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:16:29.399684 18713 external_mini_cluster.cc:1428] Setting key 560f1be0f7d65f4567adf6535ee10375
I20260806 02:16:29.404227 5972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.396280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:44973 (local address 127.18.70.126:35213)
0806 02:16:29.396417 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:29.396421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:29.397450 (+ 1029us) server_negotiation.cc:407] Connection header received
0806 02:16:29.398882 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:29.398886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:29.398962 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:29.399069 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:29.400220 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.401013 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.402430 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.402655 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.402742 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.403766 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:29.403769 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:29.403771 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:29.403783 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:29.403802 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:29.403810 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:29.403872 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:29.404004 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 02:16:29.404053 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 02:16:29.405109 6414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.396678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:35213 (local address 127.18.70.67:44973)
0806 02:16:29.397309 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:29.397351 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:29.398664 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:29.399242 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:29.399250 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:29.399359 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:29.400038 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.400058 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.401167 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.401171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.402296 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.402305 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.402787 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.402789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.402938 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.403467 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:16:29.403469 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:29.403508 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:16:29.403610 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:29.403876 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:29.403883 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:29.403888 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 02:16:29.403982 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 02:16:29.404194 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":114,"threads_started":1}
I20260806 02:16:29.406243 6412 heartbeater.cc:347] Connected to a master server at 127.18.70.126:35213
I20260806 02:16:29.406482 6412 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:29.406997 6412 heartbeater.cc:511] Master 127.18.70.126:35213 requested a full tablet report, sending...
I20260806 02:16:29.408037 5930 ts_manager.cc:194] Registered new tserver with Master: 6fec1eee14054a3bbb58cdb7be743a8b (127.18.70.67:45527)
I20260806 02:16:29.408550 5930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.70.67:44973
I20260806 02:16:29.413388 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:16:29.418717 5972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.415253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60980 (local address 127.18.70.126:35213)
0806 02:16:29.415372 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:29.415376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:29.415602 (+ 226us) server_negotiation.cc:407] Connection header received
0806 02:16:29.415715 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:29.415717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:29.415752 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:29.415816 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:29.416077 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.416541 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.417309 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.417505 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.417576 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.418309 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:29.418312 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:29.418314 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:29.418325 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:29.418342 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:29.418349 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:29.418456 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:29.418585 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 02:16:29.418618 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 02:16:29.421458 5930 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:16:29.424407 5930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:16:29.438793 6428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.433407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57556 (local address 127.18.70.65:35753)
0806 02:16:29.433805 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:29.433820 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:29.433874 (+ 54us) server_negotiation.cc:407] Connection header received
0806 02:16:29.434062 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:29.434068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:29.434311 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:29.434485 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:29.434805 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.435796 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.437284 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.438077 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.438190 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.438428 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:29.438535 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 02:16:29.438632 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":141,"threads_started":1}
I20260806 02:16:29.438800 6429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.432747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35753 (local address 127.0.0.1:57556)
0806 02:16:29.433791 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:29.433805 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:29.433905 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:29.434473 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:29.434479 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:29.434513 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:29.434651 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.434658 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.435908 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.435912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.437139 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.437149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.438212 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.438216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.438331 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.438385 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 02:16:29.438505 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 02:16:29.438629 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"thread_start_us":100,"threads_started":1}
I20260806 02:16:29.439160 6425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.432670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:45527 (local address 127.0.0.1:42066)
0806 02:16:29.432989 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:29.433032 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:29.433145 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:29.435313 (+ 2168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:29.435316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:29.435345 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:29.435429 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.435435 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.436901 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.436905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.437731 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.437739 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.438776 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.438780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.438880 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.438907 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 02:16:29.438960 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 02:16:29.439030 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260806 02:16:29.439687 6426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.433507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:44365 (local address 127.0.0.1:51660)
0806 02:16:29.434483 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:29.434495 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:29.434622 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:29.435124 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:29.435126 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:29.435145 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:29.435226 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.435231 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.437068 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.437071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.438201 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.438212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.439305 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.439310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.439397 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.439423 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 02:16:29.439478 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 02:16:29.439526 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"thread_start_us":112,"threads_started":1}
I20260806 02:16:29.439947 6427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.433315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42066 (local address 127.18.70.67:45527)
0806 02:16:29.434883 (+ 1568us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:29.434894 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:29.434916 (+ 22us) server_negotiation.cc:407] Connection header received
0806 02:16:29.434986 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:29.434991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:29.435167 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:29.435621 (+ 454us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:29.435765 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.436794 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.437902 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.438634 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.439454 (+ 820us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.439648 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:29.439704 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 02:16:29.439789 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1467,"thread_start_us":95,"threads_started":1}
I20260806 02:16:29.440482 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.434098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51660 (local address 127.18.70.66:44365)
0806 02:16:29.434404 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:29.434418 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:29.434599 (+ 181us) server_negotiation.cc:407] Connection header received
0806 02:16:29.434752 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:29.434758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:29.434966 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:29.435127 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:29.435319 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.436677 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.438350 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.439153 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.439983 (+ 830us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.440183 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:29.440233 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 02:16:29.440323 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":124,"threads_started":1}
I20260806 02:16:29.442322 6338 tablet_service.cc:1467] Processing CreateTablet for tablet 56e23f64e778458db1b7129df60732c6 (DEFAULT_TABLE table=test-table [id=10158d95994845a59ef3b686c2dde792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:29.442322 6057 tablet_service.cc:1467] Processing CreateTablet for tablet 56e23f64e778458db1b7129df60732c6 (DEFAULT_TABLE table=test-table [id=10158d95994845a59ef3b686c2dde792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:29.442617 6198 tablet_service.cc:1467] Processing CreateTablet for tablet 56e23f64e778458db1b7129df60732c6 (DEFAULT_TABLE table=test-table [id=10158d95994845a59ef3b686c2dde792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:29.443462 6338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56e23f64e778458db1b7129df60732c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:29.443462 6057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56e23f64e778458db1b7129df60732c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:29.443635 6198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56e23f64e778458db1b7129df60732c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:29.450539 6433 tablet_bootstrap.cc:492] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b: Bootstrap starting.
I20260806 02:16:29.451069 6431 tablet_bootstrap.cc:492] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2: Bootstrap starting.
I20260806 02:16:29.451887 6432 tablet_bootstrap.cc:492] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed: Bootstrap starting.
I20260806 02:16:29.452852 6431 tablet_bootstrap.cc:654] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:29.452939 6433 tablet_bootstrap.cc:654] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:29.453608 6431 log.cc:824] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:29.453723 6433 log.cc:824] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:29.453807 6432 tablet_bootstrap.cc:654] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:29.454576 6432 log.cc:824] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:29.455819 6431 tablet_bootstrap.cc:492] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2: No bootstrap required, opened a new log
I20260806 02:16:29.456009 6431 ts_tablet_manager.cc:1397] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 02:16:29.456429 6432 tablet_bootstrap.cc:492] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed: No bootstrap required, opened a new log
I20260806 02:16:29.456664 6432 ts_tablet_manager.cc:1397] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 02:16:29.457037 6433 tablet_bootstrap.cc:492] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b: No bootstrap required, opened a new log
I20260806 02:16:29.457243 6433 ts_tablet_manager.cc:1397] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 02:16:29.459915 6432 raft_consensus.cc:348] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 45527 } } peers { permanent_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35753 } } peers { permanent_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44365 } }
I20260806 02:16:29.460152 6432 raft_consensus.cc:374] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:29.460211 6432 raft_consensus.cc:729] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6157d9b6a5f345c5aa17e45b92ece6ed, State: Initialized, Role: FOLLOWER
I20260806 02:16:29.460660 6432 consensus_queue.cc:260] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 45527 } } peers { permanent_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35753 } } peers { permanent_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44365 } }
I20260806 02:16:29.460539 6431 raft_consensus.cc:348] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 45527 } } peers { permanent_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35753 } } peers { permanent_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44365 } }
I20260806 02:16:29.460799 6431 raft_consensus.cc:374] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:29.460863 6431 raft_consensus.cc:729] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146f8be44d764b4fbcbf6146ea1abdd2, State: Initialized, Role: FOLLOWER
I20260806 02:16:29.461436 6129 heartbeater.cc:503] Master 127.18.70.126:35213 was elected leader, sending a full tablet report...
I20260806 02:16:29.461412 6433 raft_consensus.cc:348] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 45527 } } peers { permanent_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35753 } } peers { permanent_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44365 } }
I20260806 02:16:29.461647 6433 raft_consensus.cc:374] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:29.461717 6433 raft_consensus.cc:729] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fec1eee14054a3bbb58cdb7be743a8b, State: Initialized, Role: FOLLOWER
I20260806 02:16:29.462121 6432 ts_tablet_manager.cc:1428] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 02:16:29.462146 6433 consensus_queue.cc:260] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 45527 } } peers { permanent_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35753 } } peers { permanent_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44365 } }
I20260806 02:16:29.462756 6431 consensus_queue.cc:260] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 45527 } } peers { permanent_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35753 } } peers { permanent_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44365 } }
I20260806 02:16:29.462816 6412 heartbeater.cc:503] Master 127.18.70.126:35213 was elected leader, sending a full tablet report...
I20260806 02:16:29.463158 6433 ts_tablet_manager.cc:1428] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 02:16:29.463686 6431 ts_tablet_manager.cc:1428] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 02:16:29.464390 6270 heartbeater.cc:503] Master 127.18.70.126:35213 was elected leader, sending a full tablet report...
W20260806 02:16:29.529480 6130 tablet.cc:2369] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:16:29.626190 6440 raft_consensus.cc:482] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:16:29.626471 6440 raft_consensus.cc:504] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 45527 } } peers { permanent_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35753 } } peers { permanent_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44365 } }
I20260806 02:16:29.627656 6440 leader_election.cc:288] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fec1eee14054a3bbb58cdb7be743a8b (127.18.70.67:45527), 6157d9b6a5f345c5aa17e45b92ece6ed (127.18.70.65:35753)
I20260806 02:16:29.631273 6272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.627937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35753 (local address 127.18.70.66:47693)
0806 02:16:29.628092 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:29.628109 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:29.628217 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:29.628510 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:29.628513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:29.628531 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:29.628652 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.628658 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.629387 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.629392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.630195 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.630203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.630835 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.630839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.630948 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.630973 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 02:16:29.631074 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 02:16:29.631142 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":1}
I20260806 02:16:29.631273 6428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.628064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:47693 (local address 127.18.70.65:35753)
0806 02:16:29.628188 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:29.628193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:29.628208 (+ 15us) server_negotiation.cc:407] Connection header received
0806 02:16:29.628352 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:29.628354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:29.628400 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:29.628487 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:29.628766 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.629268 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.630331 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.630706 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.630835 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.630887 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:29.631093 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 02:16:29.631159 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 02:16:29.631543 6439 raft_consensus.cc:482] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:16:29.631824 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.627937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:45527 (local address 127.18.70.66:56751)
0806 02:16:29.628466 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:29.628478 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:29.628607 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:29.628877 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:29.628880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:29.628895 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:29.629018 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.629023 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.629723 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.629728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.630535 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.630544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.631186 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.631190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.631285 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.631308 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 02:16:29.631595 (+ 287us) client_negotiation.cc:240] Negotiation successful
0806 02:16:29.631665 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":131,"thread_start_us":185,"threads_started":1}
I20260806 02:16:29.631760 6439 raft_consensus.cc:504] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 45527 } } peers { permanent_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35753 } } peers { permanent_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44365 } }
I20260806 02:16:29.631907 6427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.628059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:56751 (local address 127.18.70.67:45527)
0806 02:16:29.628210 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:29.628217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:29.628572 (+ 355us) server_negotiation.cc:407] Connection header received
0806 02:16:29.628724 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:29.628727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:29.628776 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:29.628857 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:29.629126 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.629607 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.630650 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.631057 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.631186 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.631242 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:29.631444 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 02:16:29.631517 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 02:16:29.631995 6078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e23f64e778458db1b7129df60732c6" candidate_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" is_pre_election: true
I20260806 02:16:29.632293 6078 raft_consensus.cc:2454] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146f8be44d764b4fbcbf6146ea1abdd2 in term 0.
I20260806 02:16:29.632416 6360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e23f64e778458db1b7129df60732c6" candidate_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" is_pre_election: true
I20260806 02:16:29.632591 6360 raft_consensus.cc:2454] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146f8be44d764b4fbcbf6146ea1abdd2 in term 0.
I20260806 02:16:29.632795 6149 leader_election.cc:302] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146f8be44d764b4fbcbf6146ea1abdd2, 6157d9b6a5f345c5aa17e45b92ece6ed; no voters:
I20260806 02:16:29.632957 6439 leader_election.cc:288] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6157d9b6a5f345c5aa17e45b92ece6ed (127.18.70.65:35753), 146f8be44d764b4fbcbf6146ea1abdd2 (127.18.70.66:44365)
I20260806 02:16:29.633417 6440 raft_consensus.cc:2781] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:16:29.633605 6440 raft_consensus.cc:482] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:16:29.633673 6440 raft_consensus.cc:3037] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:29.635010 6440 raft_consensus.cc:504] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 45527 } } peers { permanent_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35753 } } peers { permanent_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44365 } }
I20260806 02:16:29.635694 6078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e23f64e778458db1b7129df60732c6" candidate_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed"
I20260806 02:16:29.635705 6360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e23f64e778458db1b7129df60732c6" candidate_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fec1eee14054a3bbb58cdb7be743a8b"
I20260806 02:16:29.635803 6360 raft_consensus.cc:3037] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:29.635807 6078 raft_consensus.cc:3037] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:29.637050 6360 raft_consensus.cc:2454] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146f8be44d764b4fbcbf6146ea1abdd2 in term 1.
I20260806 02:16:29.637068 6078 raft_consensus.cc:2454] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146f8be44d764b4fbcbf6146ea1abdd2 in term 1.
I20260806 02:16:29.637328 6414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.633090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35753 (local address 127.18.70.67:54407)
0806 02:16:29.633199 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:29.633213 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:29.633286 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:29.633743 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:29.633746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:29.633788 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:29.633914 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.633921 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.634777 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.634780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.635931 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.635942 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.636852 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.636856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.636984 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.637017 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 02:16:29.637096 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 02:16:29.637174 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":3,"spinlock_wait_cycles":5504}
I20260806 02:16:29.637382 6149 leader_election.cc:302] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146f8be44d764b4fbcbf6146ea1abdd2, 6157d9b6a5f345c5aa17e45b92ece6ed; no voters:
I20260806 02:16:29.637492 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.633399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:36507 (local address 127.18.70.66:44365)
0806 02:16:29.633491 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:29.633496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:29.633702 (+ 206us) server_negotiation.cc:407] Connection header received
0806 02:16:29.633922 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:29.633926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:29.633974 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:29.634044 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:29.634349 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.634826 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.636265 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.636804 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.636892 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.636951 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:29.637314 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 02:16:29.637395 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 02:16:29.637615 6440 leader_election.cc:288] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [CANDIDATE]: Term 1 election: Requested vote from peers 6fec1eee14054a3bbb58cdb7be743a8b (127.18.70.67:45527), 6157d9b6a5f345c5aa17e45b92ece6ed (127.18.70.65:35753)
I20260806 02:16:29.637758 6428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.633400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:54407 (local address 127.18.70.65:35753)
0806 02:16:29.633505 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:29.633509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:29.633522 (+ 13us) server_negotiation.cc:407] Connection header received
0806 02:16:29.633560 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:29.633562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:29.633611 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:29.633697 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:29.634035 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.634663 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.636084 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.636630 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.636751 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.636792 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:29.637557 (+ 765us) server_negotiation.cc:299] Negotiation successful
0806 02:16:29.637628 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 02:16:29.637778 6440 raft_consensus.cc:2781] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:29.637861 6440 raft_consensus.cc:686] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [term 1 LEADER]: Becoming Leader. State: Replica: 146f8be44d764b4fbcbf6146ea1abdd2, State: Running, Role: LEADER
I20260806 02:16:29.637918 6442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.633216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:44365 (local address 127.18.70.67:36507)
0806 02:16:29.633617 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:29.633632 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:29.633790 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:29.634103 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:29.634108 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:29.634127 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:29.634237 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.634242 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.634945 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.634951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.636124 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:29.636137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.637058 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.637061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:29.637171 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.637193 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 02:16:29.637695 (+ 502us) client_negotiation.cc:240] Negotiation successful
0806 02:16:29.637765 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":21,"spinlock_wait_cycles":20096,"thread_start_us":149,"threads_started":1}
I20260806 02:16:29.638109 6078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e23f64e778458db1b7129df60732c6" candidate_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" is_pre_election: true
I20260806 02:16:29.638273 6078 raft_consensus.cc:2379] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6fec1eee14054a3bbb58cdb7be743a8b in current term 1: Already voted for candidate 146f8be44d764b4fbcbf6146ea1abdd2 in this term.
I20260806 02:16:29.638411 6219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56e23f64e778458db1b7129df60732c6" candidate_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" is_pre_election: true
I20260806 02:16:29.638865 6440 consensus_queue.cc:237] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 45527 } } peers { permanent_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35753 } } peers { permanent_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44365 } }
I20260806 02:16:29.638972 6289 leader_election.cc:302] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fec1eee14054a3bbb58cdb7be743a8b; no voters: 146f8be44d764b4fbcbf6146ea1abdd2, 6157d9b6a5f345c5aa17e45b92ece6ed
I20260806 02:16:29.639228 6439 raft_consensus.cc:2726] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 02:16:29.642063 5928 catalog_manager.cc:5495] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 146f8be44d764b4fbcbf6146ea1abdd2 (127.18.70.66). New cstate: current_term: 1 leader_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 45527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "146f8be44d764b4fbcbf6146ea1abdd2" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 44365 } health_report { overall_health: HEALTHY } } }
W20260806 02:16:29.645123 6413 tablet.cc:2369] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:16:29.675093 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.671407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51670 (local address 127.18.70.66:44365)
0806 02:16:29.671576 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:29.671581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:29.671599 (+ 18us) server_negotiation.cc:407] Connection header received
0806 02:16:29.671708 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:29.671710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:29.671766 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:29.671877 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:16:29.672157 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.672696 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.673551 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:29.673781 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:29.673890 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:29.674215 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:16:29.674623 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:16:29.674792 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:29.674875 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 02:16:29.674909 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 02:16:29.681183 6078 raft_consensus.cc:1261] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed [term 1 FOLLOWER]: Refusing update from remote peer 146f8be44d764b4fbcbf6146ea1abdd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:16:29.681238 6360 raft_consensus.cc:1261] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [term 1 FOLLOWER]: Refusing update from remote peer 146f8be44d764b4fbcbf6146ea1abdd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:16:29.681834 6440 consensus_queue.cc:1035] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6157d9b6a5f345c5aa17e45b92ece6ed" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 35753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:16:29.682077 6443 consensus_queue.cc:1035] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fec1eee14054a3bbb58cdb7be743a8b" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 45527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:16:29.689651 6448 mvcc.cc:205] Tried to move back new op lower bound from 7315384687326961664 to 7315384687163920384. Current Snapshot: MvccSnapshot[applied={T|T < 7315384687326961664}]
I20260806 02:16:29.690464 6447 mvcc.cc:205] Tried to move back new op lower bound from 7315384687326961664 to 7315384687163920384. Current Snapshot: MvccSnapshot[applied={T|T < 7315384687326961664 or (T in {7315384687326961664})}]
I20260806 02:16:29.695906 5930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:16:29.696066 5930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:16:29.697315 6451 mvcc.cc:205] Tried to move back new op lower bound from 7315384687326961664 to 7315384687163920384. Current Snapshot: MvccSnapshot[applied={T|T < 7315384687326961664}]
I20260806 02:16:29.698813 5930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb95976b828445cc8496bc4a022eb706: Sending DeleteTablet for 3 replicas of tablet 56e23f64e778458db1b7129df60732c6
I20260806 02:16:29.699587 6057 tablet_service.cc:1514] Processing DeleteTablet for tablet 56e23f64e778458db1b7129df60732c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:29 UTC) from {username='slave'} at 127.0.0.1:57556
I20260806 02:16:29.699818 6338 tablet_service.cc:1514] Processing DeleteTablet for tablet 56e23f64e778458db1b7129df60732c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:29 UTC) from {username='slave'} at 127.0.0.1:42066
I20260806 02:16:29.699790 6198 tablet_service.cc:1514] Processing DeleteTablet for tablet 56e23f64e778458db1b7129df60732c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:29 UTC) from {username='slave'} at 127.0.0.1:51660
I20260806 02:16:29.700130 6461 tablet_replica.cc:330] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed: stopping tablet replica
I20260806 02:16:29.700300 6337 tablet_service.cc:1514] Processing DeleteTablet for tablet 56e23f64e778458db1b7129df60732c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:29 UTC) from {username='slave'} at 127.0.0.1:42066
I20260806 02:16:29.700343 6463 tablet_replica.cc:330] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b: stopping tablet replica
I20260806 02:16:29.700551 6461 raft_consensus.cc:2229] T 56e23f64e778458db1b7129df60732c6 P 6157d9b6a5f345c5aa17e45b92ece6ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:16:29.700598 6463 raft_consensus.cc:2229] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:16:29.700876 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 5993
I20260806 02:16:29.700889 6463 raft_consensus.cc:2258] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:16:29.708720 6462 tablet_replica.cc:330] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2: stopping tablet replica
I20260806 02:16:29.709013 6462 raft_consensus.cc:2229] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:16:29.709332 6462 raft_consensus.cc:2258] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:16:29.710979 6462 ts_tablet_manager.cc:1905] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:16:29.711339 6463 ts_tablet_manager.cc:1905] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:16:29.713970 6463 ts_tablet_manager.cc:1918] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:16:29.713970 6462 ts_tablet_manager.cc:1918] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:16:29.714107 6463 log.cc:1196] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/56e23f64e778458db1b7129df60732c6
I20260806 02:16:29.714241 6462 log.cc:1196] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/56e23f64e778458db1b7129df60732c6
I20260806 02:16:29.714480 6463 ts_tablet_manager.cc:1939] T 56e23f64e778458db1b7129df60732c6 P 6fec1eee14054a3bbb58cdb7be743a8b: Deleting consensus metadata
I20260806 02:16:29.714694 6462 ts_tablet_manager.cc:1939] T 56e23f64e778458db1b7129df60732c6 P 146f8be44d764b4fbcbf6146ea1abdd2: Deleting consensus metadata
W20260806 02:16:29.715482 5915 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:35753 (error 108)
I20260806 02:16:29.715797 5916 catalog_manager.cc:4841] TS 6fec1eee14054a3bbb58cdb7be743a8b (127.18.70.67:45527): tablet 56e23f64e778458db1b7129df60732c6 (table test-table [id=10158d95994845a59ef3b686c2dde792]) successfully deleted
I20260806 02:16:29.716051 5912 catalog_manager.cc:4841] TS 146f8be44d764b4fbcbf6146ea1abdd2 (127.18.70.66:44365): tablet 56e23f64e778458db1b7129df60732c6 (table test-table [id=10158d95994845a59ef3b686c2dde792]) successfully deleted
I20260806 02:16:29.716414 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 6132
W20260806 02:16:29.716609 5916 catalog_manager.cc:4808] TS 6fec1eee14054a3bbb58cdb7be743a8b (127.18.70.67:45527): delete failed for tablet 56e23f64e778458db1b7129df60732c6 because the tablet was not found. No further retry: Not found: Tablet not found: 56e23f64e778458db1b7129df60732c6
I20260806 02:16:29.718705 6426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:29.718381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:35753 (local address 127.0.0.1:57564)
0806 02:16:29.718519 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:29.718601 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:35753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 02:16:29.718986 5915 catalog_manager.cc:4584] TS 6157d9b6a5f345c5aa17e45b92ece6ed (127.18.70.65:35753): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 56e23f64e778458db1b7129df60732c6: Network error: Client connection negotiation failed: client connection to 127.18.70.65:35753: connect: Connection refused (error 111)
I20260806 02:16:29.723745 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 6273
I20260806 02:16:29.730283 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 5896
2026-08-06T02:16:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1169 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:29 dist-test-slave-6k22 krb5kdc[6467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:29 dist-test-slave-6k22 krb5kdc[6467](info): set up 2 sockets
Aug 06 02:16:29 dist-test-slave-6k22 krb5kdc[6467](info): commencing operation
krb5kdc: starting...
W20260806 02:16:30.872527 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:30 dist-test-slave-6k22 krb5kdc[6467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:16:30.997785 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:41015
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:44903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:41015
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:31.107517 6482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:31.107769 6482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:31.107806 6482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:31.111613 6482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:31.111673 6482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:31.111691 6482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:31.111707 6482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:31.111721 6482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:31.116582 6482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:44903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:41015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:41015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:31.117511 6482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:31.118438 6482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:31.126641 6489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:31.126633 6488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:31.126633 6491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:31.126734 6482 server_base.cc:1034] running on GCE node
I20260806 02:16:31.127413 6482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:31.128466 6482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:31.129622 6482 hybrid_clock.cc:648] HybridClock initialized: now 1785982591129604 us; error 27 us; skew 500 ppm
Aug 06 02:16:31 dist-test-slave-6k22 krb5kdc[6467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982591, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:31.132359 6482 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:16:31.133260 6482 webserver.cc:460] Webserver started at http://127.18.70.126:44877/ using document root <none> and password file <none>
I20260806 02:16:31.133786 6482 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:31.133872 6482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:31.134095 6482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:31.136165 6482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "a1ce0582ac504c229b3158d671fc8002"
format_stamp: "Formatted at 2026-08-06 02:16:31 on dist-test-slave-6k22"
server_key: "adc0de8db3071dc8b8cd8c31335b59e0"
server_key_iv: "a92a8c63de4253c6dce4c3c17135e422"
server_key_version: "encryptionkey@0"
I20260806 02:16:31.136735 6482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "a1ce0582ac504c229b3158d671fc8002"
format_stamp: "Formatted at 2026-08-06 02:16:31 on dist-test-slave-6k22"
server_key: "adc0de8db3071dc8b8cd8c31335b59e0"
server_key_iv: "a92a8c63de4253c6dce4c3c17135e422"
server_key_version: "encryptionkey@0"
I20260806 02:16:31.140420 6482 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:16:31.142870 6498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:31.144091 6482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:31.144235 6482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "a1ce0582ac504c229b3158d671fc8002"
format_stamp: "Formatted at 2026-08-06 02:16:31 on dist-test-slave-6k22"
server_key: "adc0de8db3071dc8b8cd8c31335b59e0"
server_key_iv: "a92a8c63de4253c6dce4c3c17135e422"
server_key_version: "encryptionkey@0"
I20260806 02:16:31.144343 6482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:31.156891 6482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:31.160153 6482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:31.160360 6482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:31.166985 6482 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:41015
I20260806 02:16:31.167052 6556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:41015 every 8 connection(s)
I20260806 02:16:31.167927 6482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:31.170621 6557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:31.173439 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 6482
I20260806 02:16:31.173583 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:31.173868 18713 external_mini_cluster.cc:1428] Setting key 87eaf4a7992d37e292e7a61b197173ca
I20260806 02:16:31.176349 6557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002: Bootstrap starting.
I20260806 02:16:31.178850 6557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:31.179749 6557 log.cc:824] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:31.181953 6557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002: No bootstrap required, opened a new log
Aug 06 02:16:31 dist-test-slave-6k22 krb5kdc[6467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:16:31.185886 6557 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ce0582ac504c229b3158d671fc8002" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 41015 } }
I20260806 02:16:31.186165 6557 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:31.186223 6557 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1ce0582ac504c229b3158d671fc8002, State: Initialized, Role: FOLLOWER
I20260806 02:16:31.186762 6557 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ce0582ac504c229b3158d671fc8002" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 41015 } }
I20260806 02:16:31.186915 6557 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:31.186981 6557 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:31.187080 6557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:31.188133 6557 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ce0582ac504c229b3158d671fc8002" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 41015 } }
I20260806 02:16:31.188477 6557 leader_election.cc:302] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1ce0582ac504c229b3158d671fc8002; no voters:
I20260806 02:16:31.188771 6557 leader_election.cc:288] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:31.188799 6560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.175126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37220 (local address 127.18.70.126:41015)
0806 02:16:31.175525 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:31.175540 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:31.175624 (+ 84us) server_negotiation.cc:407] Connection header received
0806 02:16:31.176250 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:31.176266 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:31.176618 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:31.176886 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:31.177795 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.178767 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.179969 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.180434 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.180575 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.183076 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:31.183082 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:31.183098 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:31.183126 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:31.185596 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:31.186027 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:31.186033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:31.186038 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:31.186174 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:31.186512 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:31.186515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:31.186517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:31.187007 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:31.187258 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:31.187588 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 02:16:31.187894 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":135,"threads_started":1}
I20260806 02:16:31.188889 6562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:31.189119 6562 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [term 1 LEADER]: Becoming Leader. State: Replica: a1ce0582ac504c229b3158d671fc8002, State: Running, Role: LEADER
I20260806 02:16:31.189453 6562 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ce0582ac504c229b3158d671fc8002" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 41015 } }
I20260806 02:16:31.189963 6557 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:31.191372 6563 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1ce0582ac504c229b3158d671fc8002. Latest consensus state: current_term: 1 leader_uuid: "a1ce0582ac504c229b3158d671fc8002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ce0582ac504c229b3158d671fc8002" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 41015 } } }
I20260806 02:16:31.191529 6563 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:31.192643 6564 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1ce0582ac504c229b3158d671fc8002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1ce0582ac504c229b3158d671fc8002" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 41015 } } }
I20260806 02:16:31.192744 6564 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002 [sys.catalog]: This master's current role is: LEADER
W20260806 02:16:31.194521 6578 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:16:31.194581 6578 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:16:31.194669 6575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:31.195305 6575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:31.200145 6575 catalog_manager.cc:1347] Generated new cluster ID: 69489a6868394fbb94d0bc99f4678e18
I20260806 02:16:31.200217 6575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:31.210623 6575 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:31.211809 6575 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:31.217410 6575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002: Generated new TSK 0
I20260806 02:16:31.218050 6575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.70.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
Entry for principal kudu/127.18.70.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab.
I20260806 02:16:31.255266 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:41015
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:44903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:31.387774 6584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:31.388005 6584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:31.388074 6584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:31.392133 6584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:31.392192 6584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:31.392295 6584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:16:31.397235 6584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:44903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.65.keytab
--principal=kudu/127.18.70.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:41015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:31.398239 6584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:31.399290 6584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:31.408569 6593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:31.408550 6590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:31.408530 6591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:31.408598 6584 server_base.cc:1034] running on GCE node
I20260806 02:16:31.409255 6584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:31.409862 6584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:31.411057 6584 hybrid_clock.cc:648] HybridClock initialized: now 1785982591411042 us; error 33 us; skew 500 ppm
Aug 06 02:16:31 dist-test-slave-6k22 krb5kdc[6467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982591, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:31.413923 6584 init.cc:377] Logged in from keytab as kudu/127.18.70.65@KRBTEST.COM (short username kudu)
I20260806 02:16:31.414799 6584 webserver.cc:460] Webserver started at http://127.18.70.65:44671/ using document root <none> and password file <none>
I20260806 02:16:31.415347 6584 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:31.415438 6584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:31.415733 6584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:31.417687 6584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "1b2876a2426b4c20a2e730f8e6042fbf"
format_stamp: "Formatted at 2026-08-06 02:16:31 on dist-test-slave-6k22"
server_key: "46ea12fe3c30b98f435ad542c8ebeb2b"
server_key_iv: "877a7e5dd2a495731d2fb1e73be12a9b"
server_key_version: "encryptionkey@0"
I20260806 02:16:31.418277 6584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "1b2876a2426b4c20a2e730f8e6042fbf"
format_stamp: "Formatted at 2026-08-06 02:16:31 on dist-test-slave-6k22"
server_key: "46ea12fe3c30b98f435ad542c8ebeb2b"
server_key_iv: "877a7e5dd2a495731d2fb1e73be12a9b"
server_key_version: "encryptionkey@0"
I20260806 02:16:31.422139 6584 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 02:16:31.424516 6600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:31.425661 6584 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:31.425817 6584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "1b2876a2426b4c20a2e730f8e6042fbf"
format_stamp: "Formatted at 2026-08-06 02:16:31 on dist-test-slave-6k22"
server_key: "46ea12fe3c30b98f435ad542c8ebeb2b"
server_key_iv: "877a7e5dd2a495731d2fb1e73be12a9b"
server_key_version: "encryptionkey@0"
I20260806 02:16:31.425922 6584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:31.452466 6584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:31.455999 6584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:31.456215 6584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:31.456980 6584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:31.457928 6584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:31.457976 6584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:31.458015 6584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:31.458037 6584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:31.466939 6584 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:36387
I20260806 02:16:31.466966 6721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:36387 every 8 connection(s)
I20260806 02:16:31.467917 6584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:16:31.474408 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 6584
I20260806 02:16:31.474534 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:16:31.474840 18713 external_mini_cluster.cc:1428] Setting key 6cc038d4161a93a56970ff68e2c1c101
Aug 06 02:16:31 dist-test-slave-6k22 krb5kdc[6467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982591, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.65@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:16:31.485261 6560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.469692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:44953 (local address 127.18.70.126:41015)
0806 02:16:31.469846 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:31.469852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:31.470747 (+ 895us) server_negotiation.cc:407] Connection header received
0806 02:16:31.472608 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:31.472612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:31.472671 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:31.472777 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:31.474000 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.474778 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.476946 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.477335 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.477454 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.481195 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:31.481202 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:31.481208 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:31.481242 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:31.482920 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:31.483487 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:31.483492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:31.483494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:31.483604 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:31.483959 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:31.483966 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:31.483971 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:31.484246 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:31.484366 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:31.484876 (+ 510us) server_negotiation.cc:299] Negotiation successful
0806 02:16:31.485029 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 02:16:31.485792 6724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.470142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:41015 (local address 127.18.70.65:44953)
0806 02:16:31.470585 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:31.470630 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:31.472350 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:31.472970 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:31.472977 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:31.473379 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:31.473853 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:31.473866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.474975 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.474980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:31.476749 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:31.476761 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.477548 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.477555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:31.477795 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.478889 (+ 1094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:16:31.478917 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:31.480986 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:31.483053 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:31.483058 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:31.483070 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:31.483348 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:31.483706 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:31.483708 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:31.483710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:31.483847 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:31.484360 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:31.484365 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:31.484598 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 02:16:31.484761 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 02:16:31.485014 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":106,"threads_started":1}
I20260806 02:16:31.487358 6722 heartbeater.cc:347] Connected to a master server at 127.18.70.126:41015
I20260806 02:16:31.487632 6722 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.70.66@KRBTEST.COM; defaulting to no policy
I20260806 02:16:31.488143 6722 heartbeater.cc:511] Master 127.18.70.126:41015 requested a full tablet report, sending...
I20260806 02:16:31.489606 6516 ts_manager.cc:194] Registered new tserver with Master: 1b2876a2426b4c20a2e730f8e6042fbf (127.18.70.65:36387)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.66@KRBTEST.COM" created.
I20260806 02:16:31.491256 6516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.65@KRBTEST.COM'} at 127.18.70.65:44953
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
Entry for principal kudu/127.18.70.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab.
I20260806 02:16:31.506003 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:41015
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:44903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:31.618952 6727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:31.619189 6727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:31.619269 6727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:31.623220 6727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:31.623268 6727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:31.623322 6727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:16:31.628165 6727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:44903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.66.keytab
--principal=kudu/127.18.70.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:41015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:31.629160 6727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:31.630218 6727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:31.639854 6735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:31.639863 6733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:31.639863 6732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:31.640091 6727 server_base.cc:1034] running on GCE node
I20260806 02:16:31.640581 6727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:31.641235 6727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:31.642419 6727 hybrid_clock.cc:648] HybridClock initialized: now 1785982591642404 us; error 25 us; skew 500 ppm
Aug 06 02:16:31 dist-test-slave-6k22 krb5kdc[6467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982591, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:31.645140 6727 init.cc:377] Logged in from keytab as kudu/127.18.70.66@KRBTEST.COM (short username kudu)
I20260806 02:16:31.646032 6727 webserver.cc:460] Webserver started at http://127.18.70.66:42081/ using document root <none> and password file <none>
I20260806 02:16:31.646600 6727 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:31.646720 6727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:31.646984 6727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:31.649147 6727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "f3f7fa0a70b44a938831132b90ce3d38"
format_stamp: "Formatted at 2026-08-06 02:16:31 on dist-test-slave-6k22"
server_key: "ffaaf8e59980c8636895e1249008ff19"
server_key_iv: "2b1a1a47cc7bc35a8b0648e182475880"
server_key_version: "encryptionkey@0"
I20260806 02:16:31.649734 6727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "f3f7fa0a70b44a938831132b90ce3d38"
format_stamp: "Formatted at 2026-08-06 02:16:31 on dist-test-slave-6k22"
server_key: "ffaaf8e59980c8636895e1249008ff19"
server_key_iv: "2b1a1a47cc7bc35a8b0648e182475880"
server_key_version: "encryptionkey@0"
I20260806 02:16:31.653540 6727 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 02:16:31.655927 6742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:31.657100 6727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:31.657233 6727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "f3f7fa0a70b44a938831132b90ce3d38"
format_stamp: "Formatted at 2026-08-06 02:16:31 on dist-test-slave-6k22"
server_key: "ffaaf8e59980c8636895e1249008ff19"
server_key_iv: "2b1a1a47cc7bc35a8b0648e182475880"
server_key_version: "encryptionkey@0"
I20260806 02:16:31.657346 6727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:31.673653 6727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:31.676654 6727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:31.676862 6727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:31.677454 6727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:31.678249 6727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:31.678295 6727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:31.678382 6727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:31.678421 6727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:31.686930 6727 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:38461
I20260806 02:16:31.687000 6866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:38461 every 8 connection(s)
I20260806 02:16:31.687865 6727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:16:31.692744 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 6727
I20260806 02:16:31.692888 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:16:31.693147 18713 external_mini_cluster.cc:1428] Setting key d580d2cfb3aae24942bfcb0eba22d533
Aug 06 02:16:31 dist-test-slave-6k22 krb5kdc[6467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982591, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.66@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:16:31.702730 6560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.689767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:55775 (local address 127.18.70.126:41015)
0806 02:16:31.689916 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:31.689920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:31.690853 (+ 933us) server_negotiation.cc:407] Connection header received
0806 02:16:31.692390 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:31.692393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:31.692451 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:31.692556 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:31.693836 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.694377 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.695575 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.695800 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.695916 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.698581 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:31.698588 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:31.698594 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:31.698628 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:31.700544 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:31.701102 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:31.701107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:31.701111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:31.701212 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:31.701550 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:31.701556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:31.701559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:31.701823 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:31.701937 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:31.702394 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 02:16:31.702531 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 02:16:31.703476 6870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.690129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:41015 (local address 127.18.70.66:55775)
0806 02:16:31.690657 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:31.690739 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:31.692158 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:31.692747 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:31.692754 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:31.693158 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:31.693690 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:31.693704 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.694504 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.694507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:31.695430 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:31.695439 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.695955 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.695960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:31.696129 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.696743 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:16:31.696768 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:31.698415 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:31.700728 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:31.700733 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:31.700744 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:31.700992 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:31.701316 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:31.701318 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:31.701319 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:31.701450 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:31.701929 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:31.701934 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:31.702149 (+ 215us) client_negotiation.cc:769] Sending connection context
0806 02:16:31.702300 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 02:16:31.702533 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":149,"threads_started":1}
WARNING: no policy specified for kudu/127.18.70.67@KRBTEST.COM; defaulting to no policy
I20260806 02:16:31.704676 6868 heartbeater.cc:347] Connected to a master server at 127.18.70.126:41015
I20260806 02:16:31.704928 6868 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:31.705461 6868 heartbeater.cc:511] Master 127.18.70.126:41015 requested a full tablet report, sending...
I20260806 02:16:31.706493 6516 ts_manager.cc:194] Registered new tserver with Master: f3f7fa0a70b44a938831132b90ce3d38 (127.18.70.66:38461)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.67@KRBTEST.COM" created.
I20260806 02:16:31.707266 6516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.66@KRBTEST.COM'} at 127.18.70.66:55775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
Entry for principal kudu/127.18.70.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab.
I20260806 02:16:31.720716 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:41015
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:44903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:31.834519 6874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:31.834777 6874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:31.834816 6874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:31.839058 6874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:31.839118 6874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:31.839175 6874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:16:31.844410 6874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:44903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.67.keytab
--principal=kudu/127.18.70.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:41015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:31.845422 6874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:31.846454 6874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:31.856318 6882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:31.856462 6874 server_base.cc:1034] running on GCE node
W20260806 02:16:31.856307 6880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:31.856353 6879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:31.857079 6874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:31.857710 6874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:31.858912 6874 hybrid_clock.cc:648] HybridClock initialized: now 1785982591858883 us; error 40 us; skew 500 ppm
Aug 06 02:16:31 dist-test-slave-6k22 krb5kdc[6467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982591, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:31.861732 6874 init.cc:377] Logged in from keytab as kudu/127.18.70.67@KRBTEST.COM (short username kudu)
I20260806 02:16:31.862563 6874 webserver.cc:460] Webserver started at http://127.18.70.67:38311/ using document root <none> and password file <none>
I20260806 02:16:31.863073 6874 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:31.863140 6874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:31.863306 6874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:31.865348 6874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "b876fac22fa14e80afcef9c1f63e2e79"
format_stamp: "Formatted at 2026-08-06 02:16:31 on dist-test-slave-6k22"
server_key: "431ebf677d13ba8f9d6584e9b2cd0f7b"
server_key_iv: "d34bdd28c784a4fcc411cfc44b8f1959"
server_key_version: "encryptionkey@0"
I20260806 02:16:31.865842 6874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "b876fac22fa14e80afcef9c1f63e2e79"
format_stamp: "Formatted at 2026-08-06 02:16:31 on dist-test-slave-6k22"
server_key: "431ebf677d13ba8f9d6584e9b2cd0f7b"
server_key_iv: "d34bdd28c784a4fcc411cfc44b8f1959"
server_key_version: "encryptionkey@0"
I20260806 02:16:31.869516 6874 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:16:31.871932 6889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:31.873204 6874 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 02:16:31.873301 6874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "b876fac22fa14e80afcef9c1f63e2e79"
format_stamp: "Formatted at 2026-08-06 02:16:31 on dist-test-slave-6k22"
server_key: "431ebf677d13ba8f9d6584e9b2cd0f7b"
server_key_iv: "d34bdd28c784a4fcc411cfc44b8f1959"
server_key_version: "encryptionkey@0"
I20260806 02:16:31.873382 6874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:31.886471 6874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:31.889439 6874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:31.889631 6874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:31.890292 6874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:31.891170 6874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:31.891266 6874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:31.891343 6874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:31.891378 6874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:31.899749 6874 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:41735
I20260806 02:16:31.899783 7014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:41735 every 8 connection(s)
I20260806 02:16:31.900617 6874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:16:31.907182 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 6874
I20260806 02:16:31.907311 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:16:31.907650 18713 external_mini_cluster.cc:1428] Setting key 6934954d573990a5b74faec398e72551
Aug 06 02:16:31 dist-test-slave-6k22 krb5kdc[6467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982591, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.67@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:16:31.914825 6560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.902355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:39151 (local address 127.18.70.126:41015)
0806 02:16:31.902481 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:31.902485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:31.903290 (+ 805us) server_negotiation.cc:407] Connection header received
0806 02:16:31.904589 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:31.904592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:31.904642 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:31.904718 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:31.905786 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.906303 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.907844 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.908059 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.908178 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.910887 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:31.910891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:31.910893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:31.910918 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:31.912583 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:31.913172 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:31.913177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:31.913179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:31.913275 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:31.913671 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:31.913677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:31.913679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:31.913877 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:31.913998 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:31.914502 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 02:16:31.914628 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 02:16:31.915526 7017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.902642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:41015 (local address 127.18.70.67:39151)
0806 02:16:31.903136 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:31.903176 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:31.904396 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:31.904874 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:31.904880 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:31.905206 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:31.905662 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:31.905674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.906407 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.906410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:31.907675 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:31.907689 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.908222 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.908226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:31.908401 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.909079 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:16:31.909108 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:31.910707 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:31.912737 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:31.912744 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:31.912759 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:31.913029 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:31.913385 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:16:31.913390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:16:31.913392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:16:31.913551 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:16:31.913997 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:31.914004 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:31.914247 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 02:16:31.914440 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 02:16:31.914766 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":123,"threads_started":1}
I20260806 02:16:31.916613 7015 heartbeater.cc:347] Connected to a master server at 127.18.70.126:41015
I20260806 02:16:31.916850 7015 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:31.917301 7015 heartbeater.cc:511] Master 127.18.70.126:41015 requested a full tablet report, sending...
I20260806 02:16:31.918311 6516 ts_manager.cc:194] Registered new tserver with Master: b876fac22fa14e80afcef9c1f63e2e79 (127.18.70.67:41735)
I20260806 02:16:31.918905 6516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.70.67@KRBTEST.COM'} at 127.18.70.67:39151
I20260806 02:16:31.921674 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:16:31.930970 6560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.923793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37230 (local address 127.18.70.126:41015)
0806 02:16:31.923915 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:31.923935 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:31.924201 (+ 266us) server_negotiation.cc:407] Connection header received
0806 02:16:31.924321 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:31.924328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:31.924364 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:31.924427 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:31.925140 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.925600 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.926424 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.926614 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.926683 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.928005 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:31.928008 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:31.928010 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:31.928048 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:31.929235 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:31.929749 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:31.929752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:31.929753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:31.929823 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:31.930072 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:31.930075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:31.930076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:31.930256 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:31.930376 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:31.930658 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 02:16:31.930769 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 02:16:31.933979 6516 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:16:31.936976 6516 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:16:31.951057 7026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.944962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:38461 (local address 127.0.0.1:47050)
0806 02:16:31.945323 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:31.945373 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:31.945526 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:31.946456 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:31.946464 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:31.946491 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:31.946619 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:31.946625 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.948104 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.948108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:31.949475 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:31.949485 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.950539 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.950542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:31.950646 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.950681 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 02:16:31.950791 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 02:16:31.950909 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":1,"thread_start_us":81,"threads_started":1}
I20260806 02:16:31.951089 7027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.945093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47050 (local address 127.18.70.66:38461)
0806 02:16:31.945864 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:31.945878 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:31.945903 (+ 25us) server_negotiation.cc:407] Connection header received
0806 02:16:31.945981 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:31.945986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:31.946231 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:31.946443 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:31.946767 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.947987 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.949605 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.950411 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.950519 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.950663 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:31.950835 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 02:16:31.950935 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":153,"threads_started":1}
I20260806 02:16:31.951412 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.946026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40978 (local address 127.18.70.65:36387)
0806 02:16:31.946291 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:31.946306 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:31.946670 (+ 364us) server_negotiation.cc:407] Connection header received
0806 02:16:31.946770 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:31.946775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:31.946967 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:31.947120 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:31.947356 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.948393 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.949786 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.950426 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.950545 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.950681 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:31.951146 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 02:16:31.951247 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":95,"threads_started":1}
I20260806 02:16:31.951906 7028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.945338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:41735 (local address 127.0.0.1:49888)
0806 02:16:31.945594 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:31.945608 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:31.946420 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:31.947240 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:31.947242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:31.947255 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:31.947350 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:31.947355 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.948784 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.948787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:31.949888 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:31.949897 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.951089 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.951093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:31.951539 (+ 446us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.951608 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 02:16:31.951686 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 02:16:31.951761 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
I20260806 02:16:31.952054 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.945738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49888 (local address 127.18.70.67:41735)
0806 02:16:31.946736 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:31.946746 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:31.946770 (+ 24us) server_negotiation.cc:407] Connection header received
0806 02:16:31.946824 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:31.946828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:31.947045 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:31.947468 (+ 423us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:31.947643 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.948632 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.950013 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.950806 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.950906 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.951023 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:31.951743 (+ 720us) server_negotiation.cc:299] Negotiation successful
0806 02:16:31.951841 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":892,"thread_start_us":144,"threads_started":1}
I20260806 02:16:31.954186 6797 tablet_service.cc:1467] Processing CreateTablet for tablet f3544b239ef9461bafed230a0dab60b0 (DEFAULT_TABLE table=test-table [id=2c4b2084e7544eadb872c17cfd76eb44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:31.954552 6942 tablet_service.cc:1467] Processing CreateTablet for tablet f3544b239ef9461bafed230a0dab60b0 (DEFAULT_TABLE table=test-table [id=2c4b2084e7544eadb872c17cfd76eb44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:31.955201 6797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3544b239ef9461bafed230a0dab60b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:31.955513 6942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3544b239ef9461bafed230a0dab60b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:31.951078 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:31.945653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:36387 (local address 127.0.0.1:40978)
0806 02:16:31.946410 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:31.946423 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:31.946501 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:31.947117 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:31.947122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:31.947144 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:31.947251 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:31.947257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.948552 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.948556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:31.949652 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:31.949666 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:31.950741 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:31.950747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:31.950851 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:31.950871 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 02:16:31.950935 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 02:16:31.950986 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":77,"threads_started":1}
I20260806 02:16:31.958187 6652 tablet_service.cc:1467] Processing CreateTablet for tablet f3544b239ef9461bafed230a0dab60b0 (DEFAULT_TABLE table=test-table [id=2c4b2084e7544eadb872c17cfd76eb44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:31.959221 6652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3544b239ef9461bafed230a0dab60b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:31.962407 7035 tablet_bootstrap.cc:492] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38: Bootstrap starting.
I20260806 02:16:31.963148 7034 tablet_bootstrap.cc:492] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79: Bootstrap starting.
I20260806 02:16:31.965271 7036 tablet_bootstrap.cc:492] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf: Bootstrap starting.
I20260806 02:16:31.965420 7034 tablet_bootstrap.cc:654] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:31.965423 7035 tablet_bootstrap.cc:654] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:31.966427 7035 log.cc:824] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:31.966789 7034 log.cc:824] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:31.967321 7036 tablet_bootstrap.cc:654] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:31.968662 7035 tablet_bootstrap.cc:492] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38: No bootstrap required, opened a new log
I20260806 02:16:31.968695 7036 log.cc:824] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:31.968883 7035 ts_tablet_manager.cc:1397] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 02:16:31.969228 7034 tablet_bootstrap.cc:492] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79: No bootstrap required, opened a new log
I20260806 02:16:31.969424 7034 ts_tablet_manager.cc:1397] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 02:16:31.970716 7036 tablet_bootstrap.cc:492] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf: No bootstrap required, opened a new log
I20260806 02:16:31.970908 7036 ts_tablet_manager.cc:1397] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 02:16:31.972312 7034 raft_consensus.cc:348] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f7fa0a70b44a938831132b90ce3d38" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38461 } } peers { permanent_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36387 } } peers { permanent_uuid: "b876fac22fa14e80afcef9c1f63e2e79" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41735 } }
I20260806 02:16:31.972582 7034 raft_consensus.cc:374] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:31.972656 7034 raft_consensus.cc:729] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b876fac22fa14e80afcef9c1f63e2e79, State: Initialized, Role: FOLLOWER
I20260806 02:16:31.972965 7035 raft_consensus.cc:348] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f7fa0a70b44a938831132b90ce3d38" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38461 } } peers { permanent_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36387 } } peers { permanent_uuid: "b876fac22fa14e80afcef9c1f63e2e79" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41735 } }
I20260806 02:16:31.973137 7034 consensus_queue.cc:260] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f7fa0a70b44a938831132b90ce3d38" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38461 } } peers { permanent_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36387 } } peers { permanent_uuid: "b876fac22fa14e80afcef9c1f63e2e79" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41735 } }
I20260806 02:16:31.973241 7035 raft_consensus.cc:374] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:31.973331 7035 raft_consensus.cc:729] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3f7fa0a70b44a938831132b90ce3d38, State: Initialized, Role: FOLLOWER
I20260806 02:16:31.973763 7015 heartbeater.cc:503] Master 127.18.70.126:41015 was elected leader, sending a full tablet report...
I20260806 02:16:31.973799 7035 consensus_queue.cc:260] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f7fa0a70b44a938831132b90ce3d38" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38461 } } peers { permanent_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36387 } } peers { permanent_uuid: "b876fac22fa14e80afcef9c1f63e2e79" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41735 } }
I20260806 02:16:31.974824 7035 ts_tablet_manager.cc:1428] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260806 02:16:31.974982 7034 ts_tablet_manager.cc:1428] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 02:16:31.974931 7036 raft_consensus.cc:348] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f7fa0a70b44a938831132b90ce3d38" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38461 } } peers { permanent_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36387 } } peers { permanent_uuid: "b876fac22fa14e80afcef9c1f63e2e79" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41735 } }
I20260806 02:16:31.975200 7036 raft_consensus.cc:374] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:31.975207 6868 heartbeater.cc:503] Master 127.18.70.126:41015 was elected leader, sending a full tablet report...
I20260806 02:16:31.975268 7036 raft_consensus.cc:729] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2876a2426b4c20a2e730f8e6042fbf, State: Initialized, Role: FOLLOWER
I20260806 02:16:31.975694 7036 consensus_queue.cc:260] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f7fa0a70b44a938831132b90ce3d38" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38461 } } peers { permanent_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36387 } } peers { permanent_uuid: "b876fac22fa14e80afcef9c1f63e2e79" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41735 } }
I20260806 02:16:31.976377 6722 heartbeater.cc:503] Master 127.18.70.126:41015 was elected leader, sending a full tablet report...
I20260806 02:16:31.976871 7036 ts_tablet_manager.cc:1428] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 02:16:32.070593 7042 raft_consensus.cc:482] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:16:32.070854 7042 raft_consensus.cc:504] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f7fa0a70b44a938831132b90ce3d38" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38461 } } peers { permanent_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36387 } } peers { permanent_uuid: "b876fac22fa14e80afcef9c1f63e2e79" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41735 } }
I20260806 02:16:32.072165 7042 leader_election.cc:288] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3f7fa0a70b44a938831132b90ce3d38 (127.18.70.66:38461), b876fac22fa14e80afcef9c1f63e2e79 (127.18.70.67:41735)
I20260806 02:16:32.077757 7027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:32.072484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:59535 (local address 127.18.70.66:38461)
0806 02:16:32.072629 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:32.072634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:32.072929 (+ 295us) server_negotiation.cc:407] Connection header received
0806 02:16:32.073082 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:32.073085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:32.073141 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:32.073238 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:32.073667 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.074448 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.076450 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.077120 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.077242 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:32.077295 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:32.077558 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 02:16:32.077626 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 02:16:32.077760 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:32.072561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:52585 (local address 127.18.70.67:41735)
0806 02:16:32.072728 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:32.072731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:32.073018 (+ 287us) server_negotiation.cc:407] Connection header received
0806 02:16:32.073095 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:32.073098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:32.073141 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:32.073231 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:32.073667 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.074448 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.076450 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.077120 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.077242 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:32.077294 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:32.077557 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 02:16:32.077626 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 02:16:32.077836 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:32.072440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:41735 (local address 127.18.70.65:52585)
0806 02:16:32.072920 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:32.072932 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:32.073018 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:32.073262 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:32.073265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:32.073305 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:32.073544 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:32.073552 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.074565 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.074568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:32.076307 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:32.076319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.077242 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.077245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:32.077385 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:32.077444 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 02:16:32.077525 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 02:16:32.077648 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":375,"spinlock_wait_cycles":30848,"thread_start_us":113,"threads_started":1}
I20260806 02:16:32.077893 7044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:32.072360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:38461 (local address 127.18.70.65:59535)
0806 02:16:32.072809 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:32.072833 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:32.072951 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:32.073262 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:32.073265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:32.073307 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:32.073544 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:32.073552 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.074565 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.074568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:32.076307 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:32.076319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.077242 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.077245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:32.077381 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:32.077445 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 02:16:32.077525 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 02:16:32.077648 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":280,"spinlock_wait_cycles":6656,"thread_start_us":187,"threads_started":1}
I20260806 02:16:32.078536 6966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3544b239ef9461bafed230a0dab60b0" candidate_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876fac22fa14e80afcef9c1f63e2e79" is_pre_election: true
I20260806 02:16:32.078591 6819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3544b239ef9461bafed230a0dab60b0" candidate_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3f7fa0a70b44a938831132b90ce3d38" is_pre_election: true
I20260806 02:16:32.078858 6966 raft_consensus.cc:2454] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b2876a2426b4c20a2e730f8e6042fbf in term 0.
I20260806 02:16:32.078908 6819 raft_consensus.cc:2454] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b2876a2426b4c20a2e730f8e6042fbf in term 0.
I20260806 02:16:32.079403 6605 leader_election.cc:302] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b2876a2426b4c20a2e730f8e6042fbf, b876fac22fa14e80afcef9c1f63e2e79, f3f7fa0a70b44a938831132b90ce3d38; no voters:
I20260806 02:16:32.079718 7042 raft_consensus.cc:2781] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:16:32.079816 7042 raft_consensus.cc:482] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:16:32.079878 7042 raft_consensus.cc:3037] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:32.080915 7042 raft_consensus.cc:504] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f7fa0a70b44a938831132b90ce3d38" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38461 } } peers { permanent_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36387 } } peers { permanent_uuid: "b876fac22fa14e80afcef9c1f63e2e79" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41735 } }
I20260806 02:16:32.081287 7042 leader_election.cc:288] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [CANDIDATE]: Term 1 election: Requested vote from peers f3f7fa0a70b44a938831132b90ce3d38 (127.18.70.66:38461), b876fac22fa14e80afcef9c1f63e2e79 (127.18.70.67:41735)
I20260806 02:16:32.081604 6819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3544b239ef9461bafed230a0dab60b0" candidate_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3f7fa0a70b44a938831132b90ce3d38"
I20260806 02:16:32.081656 6966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3544b239ef9461bafed230a0dab60b0" candidate_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b876fac22fa14e80afcef9c1f63e2e79"
I20260806 02:16:32.081737 6819 raft_consensus.cc:3037] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:32.081784 6966 raft_consensus.cc:3037] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:32.082755 6819 raft_consensus.cc:2454] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b2876a2426b4c20a2e730f8e6042fbf in term 1.
I20260806 02:16:32.082845 6966 raft_consensus.cc:2454] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b2876a2426b4c20a2e730f8e6042fbf in term 1.
I20260806 02:16:32.083110 6602 leader_election.cc:302] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b2876a2426b4c20a2e730f8e6042fbf, f3f7fa0a70b44a938831132b90ce3d38; no voters:
I20260806 02:16:32.083279 7042 raft_consensus.cc:2781] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:32.083496 7042 raft_consensus.cc:686] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [term 1 LEADER]: Becoming Leader. State: Replica: 1b2876a2426b4c20a2e730f8e6042fbf, State: Running, Role: LEADER
I20260806 02:16:32.083863 7042 consensus_queue.cc:237] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f7fa0a70b44a938831132b90ce3d38" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38461 } } peers { permanent_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36387 } } peers { permanent_uuid: "b876fac22fa14e80afcef9c1f63e2e79" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41735 } }
I20260806 02:16:32.087064 6516 catalog_manager.cc:5495] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b2876a2426b4c20a2e730f8e6042fbf (127.18.70.65). New cstate: current_term: 1 leader_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f7fa0a70b44a938831132b90ce3d38" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b2876a2426b4c20a2e730f8e6042fbf" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 36387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b876fac22fa14e80afcef9c1f63e2e79" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41735 } health_report { overall_health: UNKNOWN } } }
I20260806 02:16:32.100972 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:32.097330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40990 (local address 127.18.70.65:36387)
0806 02:16:32.097443 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:32.097448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:32.097521 (+ 73us) server_negotiation.cc:407] Connection header received
0806 02:16:32.097699 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:32.097702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:32.097750 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:32.097854 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:16:32.098104 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.098619 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.099519 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.099792 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.099897 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:32.100212 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:16:32.100602 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:16:32.100738 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:32.100780 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 02:16:32.100833 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 02:16:32.106546 6819 raft_consensus.cc:1261] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38 [term 1 FOLLOWER]: Refusing update from remote peer 1b2876a2426b4c20a2e730f8e6042fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:16:32.106550 6966 raft_consensus.cc:1261] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79 [term 1 FOLLOWER]: Refusing update from remote peer 1b2876a2426b4c20a2e730f8e6042fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:16:32.107280 7046 consensus_queue.cc:1035] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3f7fa0a70b44a938831132b90ce3d38" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:16:32.107493 7042 consensus_queue.cc:1035] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b876fac22fa14e80afcef9c1f63e2e79" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:16:32.114753 7050 mvcc.cc:205] Tried to move back new op lower bound from 7315384697261592576 to 7315384697178308608. Current Snapshot: MvccSnapshot[applied={T|T < 7315384697261592576}]
I20260806 02:16:32.115479 7051 mvcc.cc:205] Tried to move back new op lower bound from 7315384697261592576 to 7315384697178308608. Current Snapshot: MvccSnapshot[applied={T|T < 7315384697261592576}]
I20260806 02:16:32.123111 6515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:16:32.123274 6515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:16:32.125730 6515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1ce0582ac504c229b3158d671fc8002: Sending DeleteTablet for 3 replicas of tablet f3544b239ef9461bafed230a0dab60b0
I20260806 02:16:32.126494 6942 tablet_service.cc:1514] Processing DeleteTablet for tablet f3544b239ef9461bafed230a0dab60b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:32 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:49888
I20260806 02:16:32.126494 6797 tablet_service.cc:1514] Processing DeleteTablet for tablet f3544b239ef9461bafed230a0dab60b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:32 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:47050
I20260806 02:16:32.126650 6652 tablet_service.cc:1514] Processing DeleteTablet for tablet f3544b239ef9461bafed230a0dab60b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:32 UTC) from {username='kudu', principal='kudu/127.18.70.126@KRBTEST.COM'} at 127.0.0.1:40978
I20260806 02:16:32.126946 7060 tablet_replica.cc:330] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79: stopping tablet replica
I20260806 02:16:32.127105 7062 tablet_replica.cc:330] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf: stopping tablet replica
I20260806 02:16:32.127297 7060 raft_consensus.cc:2229] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:16:32.127439 7062 raft_consensus.cc:2229] T f3544b239ef9461bafed230a0dab60b0 P 1b2876a2426b4c20a2e730f8e6042fbf [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:16:32.127610 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 6584
I20260806 02:16:32.127614 7060 raft_consensus.cc:2258] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:16:32.129046 7061 tablet_replica.cc:330] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38: stopping tablet replica
I20260806 02:16:32.129278 7061 raft_consensus.cc:2229] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:16:32.129478 7061 raft_consensus.cc:2258] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:16:32.130872 7060 ts_tablet_manager.cc:1905] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:16:32.130872 7061 ts_tablet_manager.cc:1905] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:16:32.133479 7060 ts_tablet_manager.cc:1918] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:16:32.133479 7061 ts_tablet_manager.cc:1918] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:16:32.133581 7060 log.cc:1196] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/f3544b239ef9461bafed230a0dab60b0
I20260806 02:16:32.133581 7061 log.cc:1196] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/f3544b239ef9461bafed230a0dab60b0
I20260806 02:16:32.133948 7061 ts_tablet_manager.cc:1939] T f3544b239ef9461bafed230a0dab60b0 P f3f7fa0a70b44a938831132b90ce3d38: Deleting consensus metadata
I20260806 02:16:32.133948 7060 ts_tablet_manager.cc:1939] T f3544b239ef9461bafed230a0dab60b0 P b876fac22fa14e80afcef9c1f63e2e79: Deleting consensus metadata
I20260806 02:16:32.135107 6502 catalog_manager.cc:4841] TS b876fac22fa14e80afcef9c1f63e2e79 (127.18.70.67:41735): tablet f3544b239ef9461bafed230a0dab60b0 (table test-table [id=2c4b2084e7544eadb872c17cfd76eb44]) successfully deleted
I20260806 02:16:32.135107 6499 catalog_manager.cc:4841] TS f3f7fa0a70b44a938831132b90ce3d38 (127.18.70.66:38461): tablet f3544b239ef9461bafed230a0dab60b0 (table test-table [id=2c4b2084e7544eadb872c17cfd76eb44]) successfully deleted
W20260806 02:16:32.137007 6499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:36387 (error 108)
I20260806 02:16:32.137276 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 6727
I20260806 02:16:32.139449 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:32.139151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:36387 (local address 127.0.0.1:40994)
0806 02:16:32.139289 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:32.139366 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:36387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 02:16:32.139698 6499 catalog_manager.cc:4584] TS 1b2876a2426b4c20a2e730f8e6042fbf (127.18.70.65:36387): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f3544b239ef9461bafed230a0dab60b0: Network error: Client connection negotiation failed: client connection to 127.18.70.65:36387: connect: Connection refused (error 111)
I20260806 02:16:32.144276 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 6874
I20260806 02:16:32.151300 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 6482
2026-08-06T02:16:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2422 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T02:16:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:32Z Disabled control of system clock
I20260806 02:16:32.185931 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:45235
--webserver_interface=127.18.70.126
--webserver_port=0
--builtin_ntp_servers=127.18.70.84:40095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:45235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 02:16:32.295256 7067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:32.295507 7067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:32.295540 7067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:32.299376 7067 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:32.299422 7067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:32.299437 7067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:32.299453 7067 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:32.299468 7067 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:32.304062 7067 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:45235
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:45235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:32.305007 7067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:32.306249 7067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:32.314502 7074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:32.314613 7075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:32.314625 7067 server_base.cc:1034] running on GCE node
W20260806 02:16:32.314771 7077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:32.315342 7067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:32.316449 7067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:32.317641 7067 hybrid_clock.cc:648] HybridClock initialized: now 1785982592317615 us; error 35 us; skew 500 ppm
I20260806 02:16:32.319458 7067 webserver.cc:460] Webserver started at http://127.18.70.126:41273/ using document root <none> and password file <none>
I20260806 02:16:32.320055 7067 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:32.320148 7067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:32.320389 7067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:32.322381 7067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "90c1fb1ba6ca4bdcaf1539e656cf8f2a"
format_stamp: "Formatted at 2026-08-06 02:16:32 on dist-test-slave-6k22"
server_key: "0f2a3b49ee5d903abc2db184158ba940"
server_key_iv: "1cfd8d7eacaf88a7ac090639b19d7468"
server_key_version: "encryptionkey@0"
I20260806 02:16:32.322928 7067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "90c1fb1ba6ca4bdcaf1539e656cf8f2a"
format_stamp: "Formatted at 2026-08-06 02:16:32 on dist-test-slave-6k22"
server_key: "0f2a3b49ee5d903abc2db184158ba940"
server_key_iv: "1cfd8d7eacaf88a7ac090639b19d7468"
server_key_version: "encryptionkey@0"
I20260806 02:16:32.326570 7067 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 02:16:32.328896 7083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:32.330030 7067 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:16:32.330168 7067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "90c1fb1ba6ca4bdcaf1539e656cf8f2a"
format_stamp: "Formatted at 2026-08-06 02:16:32 on dist-test-slave-6k22"
server_key: "0f2a3b49ee5d903abc2db184158ba940"
server_key_iv: "1cfd8d7eacaf88a7ac090639b19d7468"
server_key_version: "encryptionkey@0"
I20260806 02:16:32.330284 7067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:32.344277 7067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:32.344960 7067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:32.345196 7067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:32.352960 7067 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:45235
I20260806 02:16:32.352957 7140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:45235 every 8 connection(s)
I20260806 02:16:32.353971 7067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:32.357080 7141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:32.361264 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 7067
I20260806 02:16:32.361430 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:32.361703 18713 external_mini_cluster.cc:1428] Setting key 25001163c477ba1096079bae3fa1836a
I20260806 02:16:32.362725 7141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a: Bootstrap starting.
I20260806 02:16:32.365455 7141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:32.366340 7141 log.cc:824] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:32.368438 7141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a: No bootstrap required, opened a new log
I20260806 02:16:32.372068 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:32.363059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53358 (local address 127.18.70.126:45235)
0806 02:16:32.363726 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:32.363748 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:32.363796 (+ 48us) server_negotiation.cc:407] Connection header received
0806 02:16:32.364675 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:32.364699 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:32.365182 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:32.365569 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:32.366017 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.367296 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.368309 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.368992 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.369152 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:32.369959 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:32.369966 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:32.369983 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:32.370486 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:32.370557 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:32.370582 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:32.370675 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:32.370826 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 02:16:32.371087 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":97,"threads_started":1}
I20260806 02:16:32.373010 7141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c1fb1ba6ca4bdcaf1539e656cf8f2a" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45235 } }
I20260806 02:16:32.373238 7141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:32.373296 7141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c1fb1ba6ca4bdcaf1539e656cf8f2a, State: Initialized, Role: FOLLOWER
I20260806 02:16:32.373754 7141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c1fb1ba6ca4bdcaf1539e656cf8f2a" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45235 } }
I20260806 02:16:32.373880 7141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:32.373960 7141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:32.374073 7141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:32.374931 7141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c1fb1ba6ca4bdcaf1539e656cf8f2a" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45235 } }
I20260806 02:16:32.375200 7141 leader_election.cc:302] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90c1fb1ba6ca4bdcaf1539e656cf8f2a; no voters:
I20260806 02:16:32.375422 7141 leader_election.cc:288] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:32.375595 7146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:32.375859 7146 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [term 1 LEADER]: Becoming Leader. State: Replica: 90c1fb1ba6ca4bdcaf1539e656cf8f2a, State: Running, Role: LEADER
I20260806 02:16:32.376201 7146 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c1fb1ba6ca4bdcaf1539e656cf8f2a" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45235 } }
I20260806 02:16:32.376710 7141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:32.378047 7149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90c1fb1ba6ca4bdcaf1539e656cf8f2a. Latest consensus state: current_term: 1 leader_uuid: "90c1fb1ba6ca4bdcaf1539e656cf8f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c1fb1ba6ca4bdcaf1539e656cf8f2a" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45235 } } }
I20260806 02:16:32.378047 7148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90c1fb1ba6ca4bdcaf1539e656cf8f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c1fb1ba6ca4bdcaf1539e656cf8f2a" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 45235 } } }
I20260806 02:16:32.378212 7149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:32.378212 7148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:32.378676 7158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:32.382012 7158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:32.387607 7158 catalog_manager.cc:1347] Generated new cluster ID: c55789b0112f4f9aa89d0c054176caeb
I20260806 02:16:32.387673 7158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:32.405349 7158 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:32.406316 7158 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:32.421126 7158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a: Generated new TSK 0
I20260806 02:16:32.421844 7158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:32.434181 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.65:0
--local_ip_for_outbound_sockets=127.18.70.65
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45235
--builtin_ntp_servers=127.18.70.84:40095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 02:16:32.551405 7166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:32.551662 7166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:32.551717 7166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:32.555617 7166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:32.555667 7166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:32.555734 7166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.65
I20260806 02:16:32.560256 7166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.70.65
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.65
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:32.561276 7166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:32.562565 7166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:32.572153 7172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:32.572160 7177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:32.572268 7166 server_base.cc:1034] running on GCE node
W20260806 02:16:32.572160 7173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:32.572780 7166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:32.573378 7166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:32.574543 7166 hybrid_clock.cc:648] HybridClock initialized: now 1785982592574529 us; error 24 us; skew 500 ppm
I20260806 02:16:32.584995 7166 webserver.cc:460] Webserver started at http://127.18.70.65:41247/ using document root <none> and password file <none>
I20260806 02:16:32.585764 7166 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:32.585834 7166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:32.586074 7166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:32.588332 7166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/data/instance:
uuid: "d0efbef82f9f42c99f21911a93a92035"
format_stamp: "Formatted at 2026-08-06 02:16:32 on dist-test-slave-6k22"
server_key: "0ac8e4f795b8ee95b25cf71ceba7a764"
server_key_iv: "863c47d2c9c459931c4208983556c2c4"
server_key_version: "encryptionkey@0"
I20260806 02:16:32.588907 7166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance:
uuid: "d0efbef82f9f42c99f21911a93a92035"
format_stamp: "Formatted at 2026-08-06 02:16:32 on dist-test-slave-6k22"
server_key: "0ac8e4f795b8ee95b25cf71ceba7a764"
server_key_iv: "863c47d2c9c459931c4208983556c2c4"
server_key_version: "encryptionkey@0"
I20260806 02:16:32.592861 7166 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 02:16:32.595253 7183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:32.596530 7166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 02:16:32.596668 7166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/wal
uuid: "d0efbef82f9f42c99f21911a93a92035"
format_stamp: "Formatted at 2026-08-06 02:16:32 on dist-test-slave-6k22"
server_key: "0ac8e4f795b8ee95b25cf71ceba7a764"
server_key_iv: "863c47d2c9c459931c4208983556c2c4"
server_key_version: "encryptionkey@0"
I20260806 02:16:32.596793 7166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:32.624507 7166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:32.625202 7166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:32.625402 7166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:32.626008 7166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:32.626976 7166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:32.627046 7166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:32.627120 7166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:32.627161 7166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:32.636435 7166 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.65:41417
I20260806 02:16:32.636474 7305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.65:41417 every 8 connection(s)
I20260806 02:16:32.637321 7166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/data/info.pb
I20260806 02:16:32.642102 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 7166
I20260806 02:16:32.642234 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-0/wal/instance
I20260806 02:16:32.642529 18713 external_mini_cluster.cc:1428] Setting key 20e2ceddbf92c4bf9876dd36c18d8d4e
I20260806 02:16:32.644511 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.66:0
--local_ip_for_outbound_sockets=127.18.70.66
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45235
--builtin_ntp_servers=127.18.70.84:40095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 02:16:32.646987 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:32.639090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:56305 (local address 127.18.70.126:45235)
0806 02:16:32.639226 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:32.639230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:32.640342 (+ 1112us) server_negotiation.cc:407] Connection header received
0806 02:16:32.641741 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:32.641745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:32.641804 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:32.641890 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:32.642916 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.643631 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.644859 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.645095 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.645207 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:32.646408 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:32.646414 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:32.646418 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:32.646436 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:32.646464 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:32.646475 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:32.646557 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:32.646712 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 02:16:32.646779 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:16:32.647809 7308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:32.639490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45235 (local address 127.18.70.65:56305)
0806 02:16:32.640195 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:32.640235 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:32.641515 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:32.642075 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:32.642082 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:32.642189 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:32.642772 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:32.642787 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.643770 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.643774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:32.644735 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:32.644745 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.645248 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.645252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:32.645430 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:32.646102 (+ 672us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:16:32.646106 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:32.646165 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:16:32.646246 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:32.646563 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:32.646571 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:32.646575 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 02:16:32.646682 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 02:16:32.646919 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":133,"threads_started":1}
I20260806 02:16:32.649091 7306 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45235
I20260806 02:16:32.649355 7306 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:32.650013 7306 heartbeater.cc:511] Master 127.18.70.126:45235 requested a full tablet report, sending...
I20260806 02:16:32.651628 7102 ts_manager.cc:194] Registered new tserver with Master: d0efbef82f9f42c99f21911a93a92035 (127.18.70.65:41417)
I20260806 02:16:32.652768 7102 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.70.65:56305
W20260806 02:16:32.759747 7309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:32.760023 7309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:32.760104 7309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:32.764009 7309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:32.764089 7309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:32.764195 7309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.66
I20260806 02:16:32.768698 7309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.70.66
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.66
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:32.769711 7309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:32.771268 7309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:32.781786 7315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:32.781800 7314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:32.781824 7317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:32.782267 7309 server_base.cc:1034] running on GCE node
I20260806 02:16:32.782676 7309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:32.783399 7309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:32.784636 7309 hybrid_clock.cc:648] HybridClock initialized: now 1785982592784601 us; error 65 us; skew 500 ppm
I20260806 02:16:32.786479 7309 webserver.cc:460] Webserver started at http://127.18.70.66:41787/ using document root <none> and password file <none>
I20260806 02:16:32.787050 7309 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:32.787142 7309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:32.787406 7309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:32.789395 7309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/data/instance:
uuid: "e20ca7750f5e4919a236c54528b0676b"
format_stamp: "Formatted at 2026-08-06 02:16:32 on dist-test-slave-6k22"
server_key: "27a9da7ef919a13dd7b1ecdb01115bb5"
server_key_iv: "04853165b9e8aa571a11c012bfc4afb8"
server_key_version: "encryptionkey@0"
I20260806 02:16:32.789942 7309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance:
uuid: "e20ca7750f5e4919a236c54528b0676b"
format_stamp: "Formatted at 2026-08-06 02:16:32 on dist-test-slave-6k22"
server_key: "27a9da7ef919a13dd7b1ecdb01115bb5"
server_key_iv: "04853165b9e8aa571a11c012bfc4afb8"
server_key_version: "encryptionkey@0"
I20260806 02:16:32.793663 7309 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:16:32.795969 7324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:32.797116 7309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:32.797252 7309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/wal
uuid: "e20ca7750f5e4919a236c54528b0676b"
format_stamp: "Formatted at 2026-08-06 02:16:32 on dist-test-slave-6k22"
server_key: "27a9da7ef919a13dd7b1ecdb01115bb5"
server_key_iv: "04853165b9e8aa571a11c012bfc4afb8"
server_key_version: "encryptionkey@0"
I20260806 02:16:32.797355 7309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:32.821028 7309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:32.821767 7309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:32.821992 7309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:32.822603 7309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:32.823690 7309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:32.823770 7309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:32.823848 7309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:32.823896 7309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:32.833467 7309 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.66:45559
I20260806 02:16:32.833489 7448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.66:45559 every 8 connection(s)
I20260806 02:16:32.834456 7309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/data/info.pb
I20260806 02:16:32.841859 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 7309
I20260806 02:16:32.842041 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/wal/instance
I20260806 02:16:32.842336 18713 external_mini_cluster.cc:1428] Setting key 0d83f054d3338b17fd9bc6f12b3b719f
I20260806 02:16:32.844398 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:32.836394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.66:52985 (local address 127.18.70.126:45235)
0806 02:16:32.836530 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:32.836534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:32.837578 (+ 1044us) server_negotiation.cc:407] Connection header received
0806 02:16:32.838984 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:32.838987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:32.839046 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:32.839157 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:32.840237 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.840762 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.842042 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.842315 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.842453 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:32.843825 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:32.843832 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:32.843838 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:32.843857 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:32.843884 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:32.843896 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:32.843980 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:32.844177 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 02:16:32.844243 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 02:16:32.844818 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.70.67:0
--local_ip_for_outbound_sockets=127.18.70.67
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45235
--builtin_ntp_servers=127.18.70.84:40095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 02:16:32.845278 7451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:32.836745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45235 (local address 127.18.70.66:52985)
0806 02:16:32.837434 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:32.837474 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:32.838779 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:32.839331 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:32.839338 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:32.839417 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:32.840090 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:32.840106 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.840888 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.840892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:32.841833 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:32.841847 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:32.842488 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:32.842493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:32.842679 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:32.843394 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:16:32.843396 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:32.843445 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:16:32.843516 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:32.843992 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:32.844003 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:32.844010 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 02:16:32.844147 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 02:16:32.844361 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":128,"threads_started":1}
I20260806 02:16:32.846379 7449 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45235
I20260806 02:16:32.846612 7449 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:32.847112 7449 heartbeater.cc:511] Master 127.18.70.126:45235 requested a full tablet report, sending...
I20260806 02:16:32.848147 7102 ts_manager.cc:194] Registered new tserver with Master: e20ca7750f5e4919a236c54528b0676b (127.18.70.66:45559)
I20260806 02:16:32.848676 7102 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.70.66:52985
W20260806 02:16:32.959568 7452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:32.959857 7452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:32.959929 7452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:32.963963 7452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:32.964046 7452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:32.964128 7452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.70.67
I20260806 02:16:32.968596 7452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.70.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.70.67
--webserver_port=0
--tserver_master_addrs=127.18.70.126:45235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.70.67
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:32.969614 7452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:32.970934 7452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:32.980935 7460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:32.980937 7458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:32.980971 7457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:32.981388 7452 server_base.cc:1034] running on GCE node
I20260806 02:16:32.981768 7452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:32.982430 7452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:32.983628 7452 hybrid_clock.cc:648] HybridClock initialized: now 1785982592983607 us; error 31 us; skew 500 ppm
I20260806 02:16:32.985496 7452 webserver.cc:460] Webserver started at http://127.18.70.67:36607/ using document root <none> and password file <none>
I20260806 02:16:32.986070 7452 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:32.986156 7452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:32.986380 7452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:32.988493 7452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/data/instance:
uuid: "d999e149f5c84af39d9064f2f8fa0edb"
format_stamp: "Formatted at 2026-08-06 02:16:32 on dist-test-slave-6k22"
server_key: "ba148fffd1914707ba0b0bd05d2c6778"
server_key_iv: "45bea0a3258add7e4f6a7449d65d8d78"
server_key_version: "encryptionkey@0"
I20260806 02:16:32.989074 7452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance:
uuid: "d999e149f5c84af39d9064f2f8fa0edb"
format_stamp: "Formatted at 2026-08-06 02:16:32 on dist-test-slave-6k22"
server_key: "ba148fffd1914707ba0b0bd05d2c6778"
server_key_iv: "45bea0a3258add7e4f6a7449d65d8d78"
server_key_version: "encryptionkey@0"
I20260806 02:16:32.992842 7452 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:16:32.995211 7467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:32.996455 7452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:32.996591 7452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/wal
uuid: "d999e149f5c84af39d9064f2f8fa0edb"
format_stamp: "Formatted at 2026-08-06 02:16:32 on dist-test-slave-6k22"
server_key: "ba148fffd1914707ba0b0bd05d2c6778"
server_key_iv: "45bea0a3258add7e4f6a7449d65d8d78"
server_key_version: "encryptionkey@0"
I20260806 02:16:32.996701 7452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:33.015695 7452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:33.016350 7452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:33.016542 7452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:33.017127 7452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 02:16:33.018100 7452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 02:16:33.018170 7452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:33.018267 7452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 02:16:33.018321 7452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:33.027120 7452 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.67:41231
I20260806 02:16:33.027158 7587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.67:41231 every 8 connection(s)
I20260806 02:16:33.028043 7452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/data/info.pb
I20260806 02:16:33.032455 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 7452
I20260806 02:16:33.032579 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/wal/instance
I20260806 02:16:33.032872 18713 external_mini_cluster.cc:1428] Setting key 903ea5d5fbbb6d2d902121fa77064d52
I20260806 02:16:33.037374 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.030020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.67:52853 (local address 127.18.70.126:45235)
0806 02:16:33.030163 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:33.030166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:33.031175 (+ 1009us) server_negotiation.cc:407] Connection header received
0806 02:16:33.032615 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:33.032618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:33.032678 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:33.032790 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:33.033815 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.034406 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.035505 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.035743 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.035865 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.036874 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:33.036878 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:33.036881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:33.036892 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:33.036918 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:33.036926 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:33.037002 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:33.037177 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 02:16:33.037236 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 02:16:33.038208 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.030349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:45235 (local address 127.18.70.67:52853)
0806 02:16:33.031018 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:33.031057 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:33.032400 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:33.032991 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:33.033003 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:33.033097 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:16:33.033661 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:33.033673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.034535 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.034538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:33.035370 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:33.035379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.035904 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.035908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:33.036067 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.036628 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 02:16:33.036629 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:16:33.036665 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 02:16:33.036725 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:16:33.037004 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:16:33.037016 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:16:33.037024 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 02:16:33.037174 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 02:16:33.037457 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":132,"threads_started":1}
I20260806 02:16:33.039248 7588 heartbeater.cc:347] Connected to a master server at 127.18.70.126:45235
I20260806 02:16:33.039485 7588 heartbeater.cc:464] Registering TS with master...
I20260806 02:16:33.040002 7588 heartbeater.cc:511] Master 127.18.70.126:45235 requested a full tablet report, sending...
I20260806 02:16:33.041018 7102 ts_manager.cc:194] Registered new tserver with Master: d999e149f5c84af39d9064f2f8fa0edb (127.18.70.67:41231)
I20260806 02:16:33.041559 7102 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.70.67:52853
I20260806 02:16:33.046383 18713 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 02:16:33.051795 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.048300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53372 (local address 127.18.70.126:45235)
0806 02:16:33.048458 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:33.048462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:33.048646 (+ 184us) server_negotiation.cc:407] Connection header received
0806 02:16:33.048744 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:33.048746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:33.048782 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:33.048856 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:33.049076 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.049546 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.050321 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.050530 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.050601 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.051317 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:33.051320 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:33.051322 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:33.051353 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:33.051369 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:33.051376 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:33.051433 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:33.051528 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 02:16:33.051623 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 02:16:33.054335 7102 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 02:16:33.057283 7102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 02:16:33.070819 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.065284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:41417 (local address 127.0.0.1:58944)
0806 02:16:33.066099 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:33.066113 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:33.066210 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:33.066771 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:33.066777 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:33.066798 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:33.066912 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:33.066917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.068057 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.068062 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:33.069156 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:33.069172 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.070275 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.070279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:33.070386 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.070450 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 02:16:33.070550 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 02:16:33.070665 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":86,"threads_started":1}
I20260806 02:16:33.071280 7604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.066279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54628 (local address 127.18.70.66:45559)
0806 02:16:33.066672 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:33.066686 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:33.066733 (+ 47us) server_negotiation.cc:407] Connection header received
0806 02:16:33.066804 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:33.066809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:33.066990 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:33.067130 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:33.067352 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.068482 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.069782 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.070382 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.070493 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.070673 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:33.070755 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 02:16:33.070855 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":99,"threads_started":1}
I20260806 02:16:33.071295 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.065144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:41231 (local address 127.0.0.1:52194)
0806 02:16:33.065964 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:33.066013 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:33.066131 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:33.067052 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:33.067058 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:33.067078 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:33.067165 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:33.067171 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.068613 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.068616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:33.069736 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:33.069745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.070891 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.070894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:33.070999 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.071015 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 02:16:33.071105 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 02:16:33.071169 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":34,"thread_start_us":77,"threads_started":1}
I20260806 02:16:33.071416 7601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.065289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52194 (local address 127.18.70.67:41231)
0806 02:16:33.066529 (+ 1240us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:33.066539 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:33.066559 (+ 20us) server_negotiation.cc:407] Connection header received
0806 02:16:33.066627 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:33.066631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:33.066796 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:33.066947 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:33.067327 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.068490 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.069875 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.070639 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.070736 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.070897 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:33.071141 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 02:16:33.071233 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1149,"thread_start_us":155,"threads_started":1}
I20260806 02:16:33.071457 7602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.065646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58944 (local address 127.18.70.65:41417)
0806 02:16:33.066101 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:33.066115 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:33.066314 (+ 199us) server_negotiation.cc:407] Connection header received
0806 02:16:33.066414 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:33.066419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:33.066623 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:33.066774 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:33.067035 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.067944 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.069322 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.070137 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.070242 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.070439 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:33.071157 (+ 718us) server_negotiation.cc:299] Negotiation successful
0806 02:16:33.071268 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":96,"threads_started":1}
I20260806 02:16:33.072283 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.065869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:45559 (local address 127.0.0.1:54628)
0806 02:16:33.066233 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:33.066247 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:33.066313 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:33.067121 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:33.067124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:33.067142 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:33.067238 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:33.067244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.068657 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.068660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:33.069658 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:33.069668 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.070519 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.070523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:33.070626 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.070652 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 02:16:33.070707 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 02:16:33.070775 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":95,"threads_started":1}
I20260806 02:16:33.074151 7234 tablet_service.cc:1467] Processing CreateTablet for tablet e5b7b4fdd98140f794792c9b3fcec4c8 (DEFAULT_TABLE table=test-table [id=058594c11f40427aae96212b0c75b3d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:33.074240 7517 tablet_service.cc:1467] Processing CreateTablet for tablet e5b7b4fdd98140f794792c9b3fcec4c8 (DEFAULT_TABLE table=test-table [id=058594c11f40427aae96212b0c75b3d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:33.074571 7376 tablet_service.cc:1467] Processing CreateTablet for tablet e5b7b4fdd98140f794792c9b3fcec4c8 (DEFAULT_TABLE table=test-table [id=058594c11f40427aae96212b0c75b3d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 02:16:33.075237 7234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5b7b4fdd98140f794792c9b3fcec4c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:33.075269 7517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5b7b4fdd98140f794792c9b3fcec4c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:33.075471 7376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5b7b4fdd98140f794792c9b3fcec4c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:33.081930 7605 tablet_bootstrap.cc:492] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035: Bootstrap starting.
I20260806 02:16:33.082998 7606 tablet_bootstrap.cc:492] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb: Bootstrap starting.
I20260806 02:16:33.083873 7607 tablet_bootstrap.cc:492] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b: Bootstrap starting.
I20260806 02:16:33.084491 7605 tablet_bootstrap.cc:654] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:33.085065 7606 tablet_bootstrap.cc:654] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:33.085381 7605 log.cc:824] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:33.085660 7607 tablet_bootstrap.cc:654] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:33.085958 7606 log.cc:824] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:33.086406 7607 log.cc:824] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:33.087420 7605 tablet_bootstrap.cc:492] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035: No bootstrap required, opened a new log
I20260806 02:16:33.087677 7605 ts_tablet_manager.cc:1397] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 02:16:33.088008 7606 tablet_bootstrap.cc:492] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb: No bootstrap required, opened a new log
I20260806 02:16:33.088131 7607 tablet_bootstrap.cc:492] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b: No bootstrap required, opened a new log
I20260806 02:16:33.088204 7606 ts_tablet_manager.cc:1397] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 02:16:33.088281 7607 ts_tablet_manager.cc:1397] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 02:16:33.090875 7605 raft_consensus.cc:348] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d999e149f5c84af39d9064f2f8fa0edb" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41231 } } peers { permanent_uuid: "d0efbef82f9f42c99f21911a93a92035" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41417 } } peers { permanent_uuid: "e20ca7750f5e4919a236c54528b0676b" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45559 } }
I20260806 02:16:33.091105 7605 raft_consensus.cc:374] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:33.091166 7605 raft_consensus.cc:729] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0efbef82f9f42c99f21911a93a92035, State: Initialized, Role: FOLLOWER
I20260806 02:16:33.091660 7605 consensus_queue.cc:260] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d999e149f5c84af39d9064f2f8fa0edb" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41231 } } peers { permanent_uuid: "d0efbef82f9f42c99f21911a93a92035" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41417 } } peers { permanent_uuid: "e20ca7750f5e4919a236c54528b0676b" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45559 } }
I20260806 02:16:33.092299 7306 heartbeater.cc:503] Master 127.18.70.126:45235 was elected leader, sending a full tablet report...
I20260806 02:16:33.092478 7606 raft_consensus.cc:348] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d999e149f5c84af39d9064f2f8fa0edb" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41231 } } peers { permanent_uuid: "d0efbef82f9f42c99f21911a93a92035" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41417 } } peers { permanent_uuid: "e20ca7750f5e4919a236c54528b0676b" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45559 } }
I20260806 02:16:33.092479 7607 raft_consensus.cc:348] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d999e149f5c84af39d9064f2f8fa0edb" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41231 } } peers { permanent_uuid: "d0efbef82f9f42c99f21911a93a92035" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41417 } } peers { permanent_uuid: "e20ca7750f5e4919a236c54528b0676b" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45559 } }
I20260806 02:16:33.092681 7605 ts_tablet_manager.cc:1428] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 02:16:33.092705 7606 raft_consensus.cc:374] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:33.092713 7607 raft_consensus.cc:374] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:33.092757 7606 raft_consensus.cc:729] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d999e149f5c84af39d9064f2f8fa0edb, State: Initialized, Role: FOLLOWER
I20260806 02:16:33.092757 7607 raft_consensus.cc:729] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e20ca7750f5e4919a236c54528b0676b, State: Initialized, Role: FOLLOWER
I20260806 02:16:33.093187 7606 consensus_queue.cc:260] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d999e149f5c84af39d9064f2f8fa0edb" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41231 } } peers { permanent_uuid: "d0efbef82f9f42c99f21911a93a92035" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41417 } } peers { permanent_uuid: "e20ca7750f5e4919a236c54528b0676b" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45559 } }
I20260806 02:16:33.093187 7607 consensus_queue.cc:260] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d999e149f5c84af39d9064f2f8fa0edb" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41231 } } peers { permanent_uuid: "d0efbef82f9f42c99f21911a93a92035" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41417 } } peers { permanent_uuid: "e20ca7750f5e4919a236c54528b0676b" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45559 } }
I20260806 02:16:33.094079 7607 ts_tablet_manager.cc:1428] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 02:16:33.094079 7606 ts_tablet_manager.cc:1428] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 02:16:33.094460 7449 heartbeater.cc:503] Master 127.18.70.126:45235 was elected leader, sending a full tablet report...
I20260806 02:16:33.094715 7588 heartbeater.cc:503] Master 127.18.70.126:45235 was elected leader, sending a full tablet report...
W20260806 02:16:33.138556 7307 tablet.cc:2369] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:16:33.197009 7613 raft_consensus.cc:482] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 02:16:33.197275 7613 raft_consensus.cc:504] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d999e149f5c84af39d9064f2f8fa0edb" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41231 } } peers { permanent_uuid: "d0efbef82f9f42c99f21911a93a92035" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41417 } } peers { permanent_uuid: "e20ca7750f5e4919a236c54528b0676b" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45559 } }
I20260806 02:16:33.198474 7613 leader_election.cc:288] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d999e149f5c84af39d9064f2f8fa0edb (127.18.70.67:41231), e20ca7750f5e4919a236c54528b0676b (127.18.70.66:45559)
I20260806 02:16:33.202797 7601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.198869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:34751 (local address 127.18.70.67:41231)
0806 02:16:33.199026 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:33.199032 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:33.199138 (+ 106us) server_negotiation.cc:407] Connection header received
0806 02:16:33.199434 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:33.199438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:33.199491 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:33.199601 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:33.199887 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.200565 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.201802 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.202240 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.202368 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.202420 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:33.202591 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 02:16:33.202681 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 02:16:33.202819 7616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.198721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.67:41231 (local address 127.18.70.65:34751)
0806 02:16:33.199026 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:33.199042 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:33.199172 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:33.199626 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:33.199630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:33.199648 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:33.199768 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:33.199774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.200678 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.200683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:33.201683 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:33.201693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.202369 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.202372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:33.202457 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.202485 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 02:16:33.202575 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 02:16:33.202662 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":103,"threads_started":1}
I20260806 02:16:33.202986 7604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.199253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.70.65:49033 (local address 127.18.70.66:45559)
0806 02:16:33.199405 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:33.199410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:33.199515 (+ 105us) server_negotiation.cc:407] Connection header received
0806 02:16:33.199710 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:33.199713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:33.199766 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:33.199851 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 02:16:33.200138 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.200839 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.202024 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.202420 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.202554 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.202599 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:33.202810 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 02:16:33.202878 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 02:16:33.203001 7617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.199138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.66:45559 (local address 127.18.70.65:49033)
0806 02:16:33.199433 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:33.199445 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:16:33.199536 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:16:33.199873 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:16:33.199876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:16:33.199892 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 02:16:33.200026 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:33.200032 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.200965 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.200969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:33.201893 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:16:33.201902 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.202548 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.202552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:16:33.202668 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.202700 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 02:16:33.202792 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 02:16:33.202860 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":68,"threads_started":1}
I20260806 02:16:33.203698 7399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b7b4fdd98140f794792c9b3fcec4c8" candidate_uuid: "d0efbef82f9f42c99f21911a93a92035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20ca7750f5e4919a236c54528b0676b" is_pre_election: true
I20260806 02:16:33.203801 7539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b7b4fdd98140f794792c9b3fcec4c8" candidate_uuid: "d0efbef82f9f42c99f21911a93a92035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d999e149f5c84af39d9064f2f8fa0edb" is_pre_election: true
I20260806 02:16:33.204035 7399 raft_consensus.cc:2454] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0efbef82f9f42c99f21911a93a92035 in term 0.
I20260806 02:16:33.204095 7539 raft_consensus.cc:2454] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0efbef82f9f42c99f21911a93a92035 in term 0.
I20260806 02:16:33.204571 7187 leader_election.cc:302] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0efbef82f9f42c99f21911a93a92035, e20ca7750f5e4919a236c54528b0676b; no voters:
I20260806 02:16:33.204803 7613 raft_consensus.cc:2781] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 02:16:33.204879 7613 raft_consensus.cc:482] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 02:16:33.204909 7613 raft_consensus.cc:3037] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:33.205927 7613 raft_consensus.cc:504] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d999e149f5c84af39d9064f2f8fa0edb" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41231 } } peers { permanent_uuid: "d0efbef82f9f42c99f21911a93a92035" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41417 } } peers { permanent_uuid: "e20ca7750f5e4919a236c54528b0676b" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45559 } }
I20260806 02:16:33.206251 7613 leader_election.cc:288] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [CANDIDATE]: Term 1 election: Requested vote from peers d999e149f5c84af39d9064f2f8fa0edb (127.18.70.67:41231), e20ca7750f5e4919a236c54528b0676b (127.18.70.66:45559)
I20260806 02:16:33.206579 7539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b7b4fdd98140f794792c9b3fcec4c8" candidate_uuid: "d0efbef82f9f42c99f21911a93a92035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d999e149f5c84af39d9064f2f8fa0edb"
I20260806 02:16:33.206703 7399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b7b4fdd98140f794792c9b3fcec4c8" candidate_uuid: "d0efbef82f9f42c99f21911a93a92035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e20ca7750f5e4919a236c54528b0676b"
I20260806 02:16:33.206763 7539 raft_consensus.cc:3037] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:33.206848 7399 raft_consensus.cc:3037] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:33.207867 7539 raft_consensus.cc:2454] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0efbef82f9f42c99f21911a93a92035 in term 1.
I20260806 02:16:33.208015 7399 raft_consensus.cc:2454] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0efbef82f9f42c99f21911a93a92035 in term 1.
I20260806 02:16:33.208171 7187 leader_election.cc:302] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0efbef82f9f42c99f21911a93a92035, d999e149f5c84af39d9064f2f8fa0edb; no voters:
I20260806 02:16:33.208370 7613 raft_consensus.cc:2781] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:33.208631 7613 raft_consensus.cc:686] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [term 1 LEADER]: Becoming Leader. State: Replica: d0efbef82f9f42c99f21911a93a92035, State: Running, Role: LEADER
I20260806 02:16:33.208992 7613 consensus_queue.cc:237] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d999e149f5c84af39d9064f2f8fa0edb" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41231 } } peers { permanent_uuid: "d0efbef82f9f42c99f21911a93a92035" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41417 } } peers { permanent_uuid: "e20ca7750f5e4919a236c54528b0676b" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45559 } }
I20260806 02:16:33.211973 7100 catalog_manager.cc:5495] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0efbef82f9f42c99f21911a93a92035 (127.18.70.65). New cstate: current_term: 1 leader_uuid: "d0efbef82f9f42c99f21911a93a92035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d999e149f5c84af39d9064f2f8fa0edb" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0efbef82f9f42c99f21911a93a92035" member_type: VOTER last_known_addr { host: "127.18.70.65" port: 41417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e20ca7750f5e4919a236c54528b0676b" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45559 } health_report { overall_health: UNKNOWN } } }
I20260806 02:16:33.223809 7602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.220286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58954 (local address 127.18.70.65:41417)
0806 02:16:33.220433 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:33.220438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:33.220451 (+ 13us) server_negotiation.cc:407] Connection header received
0806 02:16:33.220608 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:33.220611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:33.220672 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:33.220785 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:16:33.221025 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.221525 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.222375 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:33.222602 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:33.222706 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:33.223013 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:16:33.223406 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:16:33.223550 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:33.223627 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 02:16:33.223674 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 02:16:33.229441 7399 raft_consensus.cc:1261] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b [term 1 FOLLOWER]: Refusing update from remote peer d0efbef82f9f42c99f21911a93a92035: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:16:33.229444 7539 raft_consensus.cc:1261] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb [term 1 FOLLOWER]: Refusing update from remote peer d0efbef82f9f42c99f21911a93a92035: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:16:33.230099 7613 consensus_queue.cc:1035] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d999e149f5c84af39d9064f2f8fa0edb" member_type: VOTER last_known_addr { host: "127.18.70.67" port: 41231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:16:33.230278 7618 consensus_queue.cc:1035] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e20ca7750f5e4919a236c54528b0676b" member_type: VOTER last_known_addr { host: "127.18.70.66" port: 45559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:16:33.238497 7622 mvcc.cc:205] Tried to move back new op lower bound from 7315384701861138432 to 7315384701787222016. Current Snapshot: MvccSnapshot[applied={T|T < 7315384701861138432}]
I20260806 02:16:33.239368 7623 mvcc.cc:205] Tried to move back new op lower bound from 7315384701861138432 to 7315384701787222016. Current Snapshot: MvccSnapshot[applied={T|T < 7315384701861138432}]
I20260806 02:16:33.245030 7101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:16:33.245214 7101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 02:16:33.247548 7101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90c1fb1ba6ca4bdcaf1539e656cf8f2a: Sending DeleteTablet for 3 replicas of tablet e5b7b4fdd98140f794792c9b3fcec4c8
I20260806 02:16:33.248409 7517 tablet_service.cc:1514] Processing DeleteTablet for tablet e5b7b4fdd98140f794792c9b3fcec4c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:33 UTC) from {username='slave'} at 127.0.0.1:52194
I20260806 02:16:33.248551 7234 tablet_service.cc:1514] Processing DeleteTablet for tablet e5b7b4fdd98140f794792c9b3fcec4c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:33 UTC) from {username='slave'} at 127.0.0.1:58944
I20260806 02:16:33.248579 7376 tablet_service.cc:1514] Processing DeleteTablet for tablet e5b7b4fdd98140f794792c9b3fcec4c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 02:16:33 UTC) from {username='slave'} at 127.0.0.1:54628
I20260806 02:16:33.248946 7632 tablet_replica.cc:330] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035: stopping tablet replica
I20260806 02:16:33.249140 7633 tablet_replica.cc:330] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b: stopping tablet replica
I20260806 02:16:33.249251 7632 raft_consensus.cc:2229] T e5b7b4fdd98140f794792c9b3fcec4c8 P d0efbef82f9f42c99f21911a93a92035 [term 1 LEADER]: Raft consensus shutting down.
I20260806 02:16:33.249480 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 7166
I20260806 02:16:33.249411 7633 raft_consensus.cc:2229] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:16:33.252655 7633 raft_consensus.cc:2258] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:16:33.252727 7631 tablet_replica.cc:330] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb: stopping tablet replica
I20260806 02:16:33.253000 7631 raft_consensus.cc:2229] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 02:16:33.253283 7631 raft_consensus.cc:2258] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 02:16:33.254199 7633 ts_tablet_manager.cc:1905] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:16:33.254611 7631 ts_tablet_manager.cc:1905] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 02:16:33.256495 7633 ts_tablet_manager.cc:1918] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:16:33.256609 7633 log.cc:1196] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-1/wal/wals/e5b7b4fdd98140f794792c9b3fcec4c8
I20260806 02:16:33.256922 7633 ts_tablet_manager.cc:1939] T e5b7b4fdd98140f794792c9b3fcec4c8 P e20ca7750f5e4919a236c54528b0676b: Deleting consensus metadata
I20260806 02:16:33.258008 7087 catalog_manager.cc:4841] TS e20ca7750f5e4919a236c54528b0676b (127.18.70.66:45559): tablet e5b7b4fdd98140f794792c9b3fcec4c8 (table test-table [id=058594c11f40427aae96212b0c75b3d7]) successfully deleted
I20260806 02:16:33.258057 7631 ts_tablet_manager.cc:1918] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 02:16:33.258147 7631 log.cc:1196] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb: Deleting WAL directory at /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785982339772911-18713-0/minicluster-data/ts-2/wal/wals/e5b7b4fdd98140f794792c9b3fcec4c8
I20260806 02:16:33.258477 7631 ts_tablet_manager.cc:1939] T e5b7b4fdd98140f794792c9b3fcec4c8 P d999e149f5c84af39d9064f2f8fa0edb: Deleting consensus metadata
I20260806 02:16:33.259694 7087 catalog_manager.cc:4841] TS d999e149f5c84af39d9064f2f8fa0edb (127.18.70.67:41231): tablet e5b7b4fdd98140f794792c9b3fcec4c8 (table test-table [id=058594c11f40427aae96212b0c75b3d7]) successfully deleted
W20260806 02:16:33.260272 7084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.70.65:41417 (error 108)
I20260806 02:16:33.260435 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 7309
I20260806 02:16:33.261905 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:33.261597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.65:41417 (local address 127.0.0.1:58964)
0806 02:16:33.261749 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 02:16:33.261819 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.65:41417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 02:16:33.262068 7084 catalog_manager.cc:4584] TS d0efbef82f9f42c99f21911a93a92035 (127.18.70.65:41417): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5b7b4fdd98140f794792c9b3fcec4c8: Network error: Client connection negotiation failed: client connection to 127.18.70.65:41417: connect: Connection refused (error 111)
I20260806 02:16:33.268841 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 7452
I20260806 02:16:33.275647 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 7067
2026-08-06T02:16:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1123 ms)
[----------] 6 tests from EncryptionPolicyTest (10887 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:33 dist-test-slave-6k22 krb5kdc[7637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:33 dist-test-slave-6k22 krb5kdc[7637](info): set up 2 sockets
Aug 06 02:16:33 dist-test-slave-6k22 krb5kdc[7637](info): commencing operation
krb5kdc: starting...
W20260806 02:16:34.336807 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:34 dist-test-slave-6k22 krb5kdc[7637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 02:16:34.458936 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36685
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:34.569893 7650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:34.570179 7650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:34.570269 7650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:34.574172 7650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:34.574251 7650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:34.574287 7650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:34.574361 7650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:34.574398 7650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:34.579499 7650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36685
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:34.580549 7650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:34.581576 7650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:34.589980 7656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:34.590157 7650 server_base.cc:1034] running on GCE node
W20260806 02:16:34.590226 7658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:34.590061 7660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:34.590823 7650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:34.591931 7650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:34.593122 7650 hybrid_clock.cc:648] HybridClock initialized: now 1785982594593094 us; error 38 us; skew 500 ppm
Aug 06 02:16:34 dist-test-slave-6k22 krb5kdc[7637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:34.595863 7650 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 02:16:34.596765 7650 webserver.cc:460] Webserver started at http://127.0.0.1:32967/ using document root <none> and password file <none>
I20260806 02:16:34.597314 7650 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:34.597404 7650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:34.597632 7650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:34.599681 7650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "0b6304d22aa3484fa8a53d1390a3eb9c"
format_stamp: "Formatted at 2026-08-06 02:16:34 on dist-test-slave-6k22"
server_key: "84abbd581733879fc7e26141bee59c5b"
server_key_iv: "1482cad21555fa7bea0c7ed272d3e2ac"
server_key_version: "encryptionkey@0"
I20260806 02:16:34.600234 7650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "0b6304d22aa3484fa8a53d1390a3eb9c"
format_stamp: "Formatted at 2026-08-06 02:16:34 on dist-test-slave-6k22"
server_key: "84abbd581733879fc7e26141bee59c5b"
server_key_iv: "1482cad21555fa7bea0c7ed272d3e2ac"
server_key_version: "encryptionkey@0"
I20260806 02:16:34.603811 7650 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:16:34.606128 7668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:34.607327 7650 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:16:34.607465 7650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "0b6304d22aa3484fa8a53d1390a3eb9c"
format_stamp: "Formatted at 2026-08-06 02:16:34 on dist-test-slave-6k22"
server_key: "84abbd581733879fc7e26141bee59c5b"
server_key_iv: "1482cad21555fa7bea0c7ed272d3e2ac"
server_key_version: "encryptionkey@0"
I20260806 02:16:34.607599 7650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:34.628176 7650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:34.630884 7650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:34.631090 7650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:34.638113 7650 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36685
I20260806 02:16:34.638164 7724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36685 every 8 connection(s)
I20260806 02:16:34.639072 7650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:34.641773 7726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:34.644905 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 7650
I20260806 02:16:34.645030 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:34.645291 18713 external_mini_cluster.cc:1428] Setting key ae8197723d19adb5edc84b6b94cfb671
I20260806 02:16:34.648022 7726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c: Bootstrap starting.
I20260806 02:16:34.650728 7726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:34.651633 7726 log.cc:824] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:34.653828 7726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c: No bootstrap required, opened a new log
Aug 06 02:16:34 dist-test-slave-6k22 krb5kdc[7637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 02:16:34.658067 7726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6304d22aa3484fa8a53d1390a3eb9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36685 } }
I20260806 02:16:34.658342 7726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:34.658468 7726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6304d22aa3484fa8a53d1390a3eb9c, State: Initialized, Role: FOLLOWER
I20260806 02:16:34.658967 7726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6304d22aa3484fa8a53d1390a3eb9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36685 } }
I20260806 02:16:34.659139 7726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:34.659207 7726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:34.659315 7726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:34.660421 7726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6304d22aa3484fa8a53d1390a3eb9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36685 } }
I20260806 02:16:34.660813 7726 leader_election.cc:302] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b6304d22aa3484fa8a53d1390a3eb9c; no voters:
I20260806 02:16:34.661136 7726 leader_election.cc:288] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:34.661154 7730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:34.646780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59039 (local address 127.0.0.1:36685)
0806 02:16:34.647204 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:34.647224 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:34.647267 (+ 43us) server_negotiation.cc:407] Connection header received
0806 02:16:34.648125 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:34.648149 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:34.648579 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:34.648933 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:34.649628 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:34.651116 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:34.652214 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:34.652845 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:34.652994 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:34.655446 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:34.655452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:34.655467 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:34.655496 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:34.657507 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:34.658011 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:34.658016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:34.658021 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:34.658162 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:34.658738 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:34.658741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:34.658743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:34.659279 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:34.659524 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:34.659901 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 02:16:34.660130 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":114,"threads_started":1}
I20260806 02:16:34.661253 7732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:34.661466 7732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [term 1 LEADER]: Becoming Leader. State: Replica: 0b6304d22aa3484fa8a53d1390a3eb9c, State: Running, Role: LEADER
I20260806 02:16:34.661815 7732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6304d22aa3484fa8a53d1390a3eb9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36685 } }
I20260806 02:16:34.662438 7726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:34.663482 7734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b6304d22aa3484fa8a53d1390a3eb9c. Latest consensus state: current_term: 1 leader_uuid: "0b6304d22aa3484fa8a53d1390a3eb9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6304d22aa3484fa8a53d1390a3eb9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36685 } } }
I20260806 02:16:34.663602 7733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b6304d22aa3484fa8a53d1390a3eb9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b6304d22aa3484fa8a53d1390a3eb9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36685 } } }
I20260806 02:16:34.663676 7734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:34.663697 7733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:34.664510 7742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:34.664659 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:34.667511 7742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:34.672703 7742 catalog_manager.cc:1347] Generated new cluster ID: 6d0808f4dcb34ff387cebb1963cf8ecd
I20260806 02:16:34.672820 7742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:34.679943 7730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:34.668034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42895 (local address 127.0.0.1:36685)
0806 02:16:34.668136 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:34.668140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:34.671367 (+ 3227us) server_negotiation.cc:407] Connection header received
0806 02:16:34.671508 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:34.671511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:34.671617 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:34.671705 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:34.672523 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:34.673248 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:34.674419 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:34.674812 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:34.674924 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:34.676468 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:34.676472 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:34.676475 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:34.676499 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:34.678107 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:34.678631 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:34.678634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:34.678636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:34.678728 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:34.679054 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:34.679057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:34.679059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:34.679255 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:34.679396 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:34.679667 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 02:16:34.679775 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 02:16:34.684455 7742 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:34.685263 7742 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:34.691774 7742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b6304d22aa3484fa8a53d1390a3eb9c: Generated new TSK 0
I20260806 02:16:34.692368 7742 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:34.698921 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 7650
W20260806 02:16:34.706969 7635 connection.cc:535] client connection to 127.0.0.1:36685 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36685): Connection reset by peer (error 104)
2026-08-06T02:16:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:34 dist-test-slave-6k22 krb5kdc[7759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:34 dist-test-slave-6k22 krb5kdc[7759](info): set up 2 sockets
Aug 06 02:16:34 dist-test-slave-6k22 krb5kdc[7759](info): commencing operation
krb5kdc: starting...
W20260806 02:16:35.837242 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:35 dist-test-slave-6k22 krb5kdc[7759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 02:16:35.968950 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41447
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:36.081220 7773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:36.081506 7773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:36.081545 7773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:36.085816 7773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:36.085870 7773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:36.085888 7773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:36.085904 7773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:36.085919 7773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:36.090975 7773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41447
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:36.092058 7773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:36.093057 7773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:36.101809 7780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:36.101915 7782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:36.101815 7779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:36.102056 7773 server_base.cc:1034] running on GCE node
I20260806 02:16:36.102596 7773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:36.103719 7773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:36.104899 7773 hybrid_clock.cc:648] HybridClock initialized: now 1785982596104883 us; error 31 us; skew 500 ppm
Aug 06 02:16:36 dist-test-slave-6k22 krb5kdc[7759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982596, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:36.107702 7773 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 02:16:36.108567 7773 webserver.cc:460] Webserver started at http://127.0.0.1:41471/ using document root <none> and password file <none>
I20260806 02:16:36.109149 7773 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:36.109248 7773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:36.109508 7773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:36.111536 7773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "abacac7a87f247e685918322214dddeb"
format_stamp: "Formatted at 2026-08-06 02:16:36 on dist-test-slave-6k22"
server_key: "706871d6db67e27361912c4f27d25181"
server_key_iv: "31c3277d39546cda35b6312dfe347f47"
server_key_version: "encryptionkey@0"
I20260806 02:16:36.112171 7773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "abacac7a87f247e685918322214dddeb"
format_stamp: "Formatted at 2026-08-06 02:16:36 on dist-test-slave-6k22"
server_key: "706871d6db67e27361912c4f27d25181"
server_key_iv: "31c3277d39546cda35b6312dfe347f47"
server_key_version: "encryptionkey@0"
I20260806 02:16:36.115908 7773 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:16:36.118413 7790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:36.119670 7773 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:36.119810 7773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "abacac7a87f247e685918322214dddeb"
format_stamp: "Formatted at 2026-08-06 02:16:36 on dist-test-slave-6k22"
server_key: "706871d6db67e27361912c4f27d25181"
server_key_iv: "31c3277d39546cda35b6312dfe347f47"
server_key_version: "encryptionkey@0"
I20260806 02:16:36.119923 7773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:36.162593 7773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:36.165985 7773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:36.166141 7773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:36.173390 7773 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41447
I20260806 02:16:36.173450 7845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41447 every 8 connection(s)
I20260806 02:16:36.174790 7773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:36.175886 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 7773
I20260806 02:16:36.176059 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:36.176416 18713 external_mini_cluster.cc:1428] Setting key 5a425bfcf14dc8594bbb06650df87bab
I20260806 02:16:36.178586 7846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:36.185204 7846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb: Bootstrap starting.
I20260806 02:16:36.186280 7849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:36.177894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33809 (local address 127.0.0.1:41447)
0806 02:16:36.178354 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:36.178394 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:36.178462 (+ 68us) server_negotiation.cc:407] Connection header received
0806 02:16:36.179305 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:36.179328 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:36.179773 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:36.180131 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:36.180942 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:36.181921 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:36.183126 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:36.183644 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:36.183773 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:36.184676 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:36.184681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:36.184694 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:36.185051 (+ 357us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:36.185119 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:36.185141 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:36.185251 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:36.185454 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 02:16:36.185610 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":145,"threads_started":1}
I20260806 02:16:36.187445 7846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:36.188236 7846 log.cc:824] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:36.189263 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:36.190204 7846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb: No bootstrap required, opened a new log
I20260806 02:16:36.195211 7846 raft_consensus.cc:348] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abacac7a87f247e685918322214dddeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } }
I20260806 02:16:36.195513 7846 raft_consensus.cc:374] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:36.195622 7846 raft_consensus.cc:729] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abacac7a87f247e685918322214dddeb, State: Initialized, Role: FOLLOWER
I20260806 02:16:36.195782 7849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:36.191282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38421 (local address 127.0.0.1:41447)
0806 02:16:36.191392 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:36.191395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:36.191828 (+ 433us) server_negotiation.cc:407] Connection header received
0806 02:16:36.191927 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:36.191929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:36.191963 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:36.192025 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:36.192405 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:36.192869 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:36.194066 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:36.194277 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:36.194356 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:36.195181 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:36.195185 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:36.195187 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:36.195197 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:36.195217 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:36.195226 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:36.195290 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:36.195454 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 02:16:36.195500 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 02:16:36.196161 7846 consensus_queue.cc:260] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abacac7a87f247e685918322214dddeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } }
I20260806 02:16:36.196322 7846 raft_consensus.cc:388] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:36.196389 7846 raft_consensus.cc:482] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:36.196483 7846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:36.197430 7846 raft_consensus.cc:504] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abacac7a87f247e685918322214dddeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } }
I20260806 02:16:36.197744 7846 leader_election.cc:302] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abacac7a87f247e685918322214dddeb; no voters:
I20260806 02:16:36.197976 7846 leader_election.cc:288] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:36.198091 7857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:36.198387 7857 raft_consensus.cc:686] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [term 1 LEADER]: Becoming Leader. State: Replica: abacac7a87f247e685918322214dddeb, State: Running, Role: LEADER
I20260806 02:16:36.198719 7857 consensus_queue.cc:237] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abacac7a87f247e685918322214dddeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } }
I20260806 02:16:36.199247 7846 sys_catalog.cc:564] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:36.200896 7859 sys_catalog.cc:455] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader abacac7a87f247e685918322214dddeb. Latest consensus state: current_term: 1 leader_uuid: "abacac7a87f247e685918322214dddeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abacac7a87f247e685918322214dddeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } } }
I20260806 02:16:36.201052 7859 sys_catalog.cc:458] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:36.201399 7858 sys_catalog.cc:455] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abacac7a87f247e685918322214dddeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abacac7a87f247e685918322214dddeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41447 } } }
I20260806 02:16:36.201604 7858 sys_catalog.cc:458] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:36.201809 7874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:36.204633 7874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:36.209731 7874 catalog_manager.cc:1347] Generated new cluster ID: 91727f0eb6f84c199721f05c6fb356a1
I20260806 02:16:36.209792 7874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:36.227629 7874 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:36.228780 7874 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:36.239238 7874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abacac7a87f247e685918322214dddeb: Generated new TSK 0
I20260806 02:16:36.240006 7874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:36.249145 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 7773
W20260806 02:16:36.257477 7757 connection.cc:535] client connection to 127.0.0.1:41447 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41447): Connection reset by peer (error 104)
2026-08-06T02:16:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1550 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:36 dist-test-slave-6k22 krb5kdc[7883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:36 dist-test-slave-6k22 krb5kdc[7883](info): set up 2 sockets
Aug 06 02:16:36 dist-test-slave-6k22 krb5kdc[7883](info): commencing operation
krb5kdc: starting...
W20260806 02:16:37.313045 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:37 dist-test-slave-6k22 krb5kdc[7883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 02:16:37.438084 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44125
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:37.550163 7898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:37.550432 7898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:37.550474 7898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:37.554522 7898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:37.554579 7898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:37.554598 7898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:37.554632 7898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:37.554647 7898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:37.559973 7898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44125
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:37.560974 7898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:37.561959 7898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:37.570823 7906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:37.570945 7904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:37.570896 7903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:37.571038 7898 server_base.cc:1034] running on GCE node
I20260806 02:16:37.571843 7898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:37.572949 7898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:37.574106 7898 hybrid_clock.cc:648] HybridClock initialized: now 1785982597574096 us; error 26 us; skew 500 ppm
Aug 06 02:16:37 dist-test-slave-6k22 krb5kdc[7883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982597, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:37.576921 7898 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 02:16:37.577732 7898 webserver.cc:460] Webserver started at http://127.0.0.1:32825/ using document root <none> and password file <none>
I20260806 02:16:37.578263 7898 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:37.578328 7898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:37.578507 7898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:37.580607 7898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "6d78119929ce47ec86d9fe133245750b"
format_stamp: "Formatted at 2026-08-06 02:16:37 on dist-test-slave-6k22"
server_key: "d175dc1d06e7a9aaf6cdeafe61cd0bf7"
server_key_iv: "ce3a789f793615f0acbc8fa663f5e19f"
server_key_version: "encryptionkey@0"
I20260806 02:16:37.581116 7898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "6d78119929ce47ec86d9fe133245750b"
format_stamp: "Formatted at 2026-08-06 02:16:37 on dist-test-slave-6k22"
server_key: "d175dc1d06e7a9aaf6cdeafe61cd0bf7"
server_key_iv: "ce3a789f793615f0acbc8fa663f5e19f"
server_key_version: "encryptionkey@0"
I20260806 02:16:37.584736 7898 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:16:37.587011 7913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:37.588268 7898 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:37.588418 7898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "6d78119929ce47ec86d9fe133245750b"
format_stamp: "Formatted at 2026-08-06 02:16:37 on dist-test-slave-6k22"
server_key: "d175dc1d06e7a9aaf6cdeafe61cd0bf7"
server_key_iv: "ce3a789f793615f0acbc8fa663f5e19f"
server_key_version: "encryptionkey@0"
I20260806 02:16:37.588536 7898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:37.593042 7898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:37.601917 7898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:37.602128 7898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:37.609365 7898 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44125
I20260806 02:16:37.609380 7971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44125 every 8 connection(s)
I20260806 02:16:37.610299 7898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:37.613116 7972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:37.614166 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 7898
I20260806 02:16:37.614305 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:37.614660 18713 external_mini_cluster.cc:1428] Setting key fb5ff6372ccd8380dce7c0d44be721dd
I20260806 02:16:37.619639 7972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b: Bootstrap starting.
I20260806 02:16:37.619918 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:37.616100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43025 (local address 127.0.0.1:44125)
0806 02:16:37.616522 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:37.616536 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:37.616570 (+ 34us) server_negotiation.cc:407] Connection header received
0806 02:16:37.617160 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:37.617176 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:37.617458 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:37.617801 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:37.618633 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:37.618638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:37.618650 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:37.618660 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:37.618714 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:37.618732 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:37.618882 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:37.618954 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 02:16:37.619111 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":130,"threads_started":1}
I20260806 02:16:37.622155 7972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:37.622864 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:37.622991 7972 log.cc:824] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:37.625087 7972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b: No bootstrap required, opened a new log
I20260806 02:16:37.629001 7972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d78119929ce47ec86d9fe133245750b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44125 } }
I20260806 02:16:37.629318 7976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:37.624997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56801 (local address 127.0.0.1:44125)
0806 02:16:37.625114 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:37.625117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:37.627818 (+ 2701us) server_negotiation.cc:407] Connection header received
0806 02:16:37.628075 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:37.628080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:37.628142 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:37.628232 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:37.628944 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:37.628947 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:37.628949 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:37.628959 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:37.628981 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:37.628991 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:37.629059 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:37.629170 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 02:16:37.629197 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 02:16:37.629335 7972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:37.629477 7972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d78119929ce47ec86d9fe133245750b, State: Initialized, Role: FOLLOWER
I20260806 02:16:37.629932 7972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d78119929ce47ec86d9fe133245750b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44125 } }
I20260806 02:16:37.630084 7972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:37.630155 7972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:37.630263 7972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:37.631081 7972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d78119929ce47ec86d9fe133245750b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44125 } }
I20260806 02:16:37.631413 7972 leader_election.cc:302] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d78119929ce47ec86d9fe133245750b; no voters:
I20260806 02:16:37.631729 7972 leader_election.cc:288] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:37.631847 7983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:37.632090 7983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [term 1 LEADER]: Becoming Leader. State: Replica: 6d78119929ce47ec86d9fe133245750b, State: Running, Role: LEADER
I20260806 02:16:37.632442 7983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d78119929ce47ec86d9fe133245750b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44125 } }
I20260806 02:16:37.632941 7972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:37.634044 7985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d78119929ce47ec86d9fe133245750b. Latest consensus state: current_term: 1 leader_uuid: "6d78119929ce47ec86d9fe133245750b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d78119929ce47ec86d9fe133245750b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44125 } } }
I20260806 02:16:37.634217 7984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d78119929ce47ec86d9fe133245750b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d78119929ce47ec86d9fe133245750b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44125 } } }
I20260806 02:16:37.634202 7985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:37.634493 7984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:37.634723 7992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:37.637831 7992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:37.643074 7992 catalog_manager.cc:1347] Generated new cluster ID: 31d25fc7030944b08b11285b13762013
I20260806 02:16:37.643186 7992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:37.665750 7992 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:37.666074 7992 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:37.674921 7992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d78119929ce47ec86d9fe133245750b: Generated new TSK 0
I20260806 02:16:37.675540 7992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:37.685751 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 7898
2026-08-06T02:16:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1436 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:37 dist-test-slave-6k22 krb5kdc[8004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:37 dist-test-slave-6k22 krb5kdc[8004](info): set up 2 sockets
Aug 06 02:16:37 dist-test-slave-6k22 krb5kdc[8004](info): commencing operation
krb5kdc: starting...
W20260806 02:16:38.745184 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:38 dist-test-slave-6k22 krb5kdc[8004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 02:16:38.868571 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:38.977268 8017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:38.977530 8017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:38.977603 8017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:38.981545 8017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:38.981627 8017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:38.981664 8017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:38.981716 8017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:38.981756 8017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:38.986829 8017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:38.987831 8017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:38.988797 8017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:38.997119 8023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:38.997103 8022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:38.997368 8017 server_base.cc:1034] running on GCE node
W20260806 02:16:38.997094 8025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:38.997998 8017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:38.999100 8017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:39.000284 8017 hybrid_clock.cc:648] HybridClock initialized: now 1785982599000256 us; error 39 us; skew 500 ppm
Aug 06 02:16:38 dist-test-slave-6k22 krb5kdc[8004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982598, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:39.003268 8017 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 02:16:39.004287 8017 webserver.cc:460] Webserver started at http://127.0.0.1:43379/ using document root <none> and password file <none>
I20260806 02:16:39.004788 8017 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:39.004853 8017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:39.005012 8017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:39.007088 8017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "26234d381b194071b645ec8369f4bd7f"
format_stamp: "Formatted at 2026-08-06 02:16:39 on dist-test-slave-6k22"
server_key: "3bfbb758f06ac9d4c64e5a83d912b409"
server_key_iv: "fdb0507c65d876075b5719d20b4f9545"
server_key_version: "encryptionkey@0"
I20260806 02:16:39.007647 8017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "26234d381b194071b645ec8369f4bd7f"
format_stamp: "Formatted at 2026-08-06 02:16:39 on dist-test-slave-6k22"
server_key: "3bfbb758f06ac9d4c64e5a83d912b409"
server_key_iv: "fdb0507c65d876075b5719d20b4f9545"
server_key_version: "encryptionkey@0"
I20260806 02:16:39.011237 8017 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 02:16:39.013489 8033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:39.014708 8017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:16:39.014814 8017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "26234d381b194071b645ec8369f4bd7f"
format_stamp: "Formatted at 2026-08-06 02:16:39 on dist-test-slave-6k22"
server_key: "3bfbb758f06ac9d4c64e5a83d912b409"
server_key_iv: "fdb0507c65d876075b5719d20b4f9545"
server_key_version: "encryptionkey@0"
I20260806 02:16:39.014904 8017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:39.026865 8017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:39.029615 8017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:39.029865 8017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:39.037854 8017 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42201
I20260806 02:16:39.037871 8091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42201 every 8 connection(s)
I20260806 02:16:39.038816 8017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:39.041531 8092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:39.044600 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 8017
I20260806 02:16:39.044734 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:39.045001 18713 external_mini_cluster.cc:1428] Setting key 11d19d72da40e3feec6470a9f3389e23
I20260806 02:16:39.047232 8092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f: Bootstrap starting.
I20260806 02:16:39.049760 8092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:39.050593 8092 log.cc:824] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:39.052759 8092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f: No bootstrap required, opened a new log
Aug 06 02:16:39 dist-test-slave-6k22 krb5kdc[8004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 02:16:39.056319 8092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26234d381b194071b645ec8369f4bd7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42201 } }
I20260806 02:16:39.056545 8092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:39.056627 8092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26234d381b194071b645ec8369f4bd7f, State: Initialized, Role: FOLLOWER
I20260806 02:16:39.057102 8092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26234d381b194071b645ec8369f4bd7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42201 } }
I20260806 02:16:39.057284 8092 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:39.057358 8092 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:39.057449 8092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:39.058564 8092 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26234d381b194071b645ec8369f4bd7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42201 } }
I20260806 02:16:39.058912 8092 leader_election.cc:302] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26234d381b194071b645ec8369f4bd7f; no voters:
I20260806 02:16:39.059234 8092 leader_election.cc:288] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:39.059319 8097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:39.059396 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:39.046373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39319 (local address 127.0.0.1:42201)
0806 02:16:39.046754 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:39.046769 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:39.046804 (+ 35us) server_negotiation.cc:407] Connection header received
0806 02:16:39.047502 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:39.047519 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:39.047908 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:39.048166 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:39.049096 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:39.050118 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:39.051234 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:39.051755 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:39.051880 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:39.054524 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:39.054529 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:39.054541 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:39.054566 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:39.056260 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:39.056719 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:39.056731 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:39.056736 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:39.056843 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:39.057177 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:39.057179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:39.057180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:39.057699 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:39.057839 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:39.058180 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 02:16:39.058480 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":124,"threads_started":1}
I20260806 02:16:39.059487 8097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [term 1 LEADER]: Becoming Leader. State: Replica: 26234d381b194071b645ec8369f4bd7f, State: Running, Role: LEADER
I20260806 02:16:39.059831 8097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26234d381b194071b645ec8369f4bd7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42201 } }
I20260806 02:16:39.060490 8092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:39.061735 8099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26234d381b194071b645ec8369f4bd7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26234d381b194071b645ec8369f4bd7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42201 } } }
I20260806 02:16:39.061862 8099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:39.062584 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 02:16:39.066067 8115 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:16:39.066196 8115 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:16:39.063024 8100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26234d381b194071b645ec8369f4bd7f. Latest consensus state: current_term: 1 leader_uuid: "26234d381b194071b645ec8369f4bd7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26234d381b194071b645ec8369f4bd7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42201 } } }
I20260806 02:16:39.066323 8100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:39.066671 8107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:39.067411 8107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:39.072626 8107 catalog_manager.cc:1347] Generated new cluster ID: f48cb7ea39ad4b1294ad0e585019f65a
I20260806 02:16:39.072727 8107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:39.073756 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:39.065110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49427 (local address 127.0.0.1:42201)
0806 02:16:39.065200 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:39.065204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:39.065217 (+ 13us) server_negotiation.cc:407] Connection header received
0806 02:16:39.065249 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:39.065252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:39.065288 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:39.065379 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:39.066184 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:39.066942 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:39.068055 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:39.068379 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:39.068480 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:39.070089 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:39.070092 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:39.070095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:39.070122 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:39.071815 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:39.072290 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:39.072294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:39.072295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:39.072375 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:39.072675 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:39.072678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:39.072680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:39.072891 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:39.072992 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:39.073354 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 02:16:39.073616 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":34}
I20260806 02:16:39.081553 8107 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:39.082715 8107 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:39.092496 8107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26234d381b194071b645ec8369f4bd7f: Generated new TSK 0
I20260806 02:16:39.093264 8107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:39.102013 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 8017
2026-08-06T02:16:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:39 dist-test-slave-6k22 krb5kdc[8126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:39 dist-test-slave-6k22 krb5kdc[8126](info): set up 2 sockets
Aug 06 02:16:39 dist-test-slave-6k22 krb5kdc[8126](info): commencing operation
krb5kdc: starting...
W20260806 02:16:40.156087 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:40 dist-test-slave-6k22 krb5kdc[8126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 02:16:40.282218 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34589
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:40.392616 8139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:40.392932 8139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:40.392974 8139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:40.396857 8139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:40.396919 8139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:40.396937 8139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:40.396970 8139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:40.396986 8139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:40.402088 8139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:40.403028 8139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:40.403968 8139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:40.412477 8144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:40.412459 8145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:40.412449 8147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:40.412683 8139 server_base.cc:1034] running on GCE node
I20260806 02:16:40.413268 8139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:40.414284 8139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:40.415446 8139 hybrid_clock.cc:648] HybridClock initialized: now 1785982600415429 us; error 31 us; skew 500 ppm
Aug 06 02:16:40 dist-test-slave-6k22 krb5kdc[8126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982600, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:40.418226 8139 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 02:16:40.419137 8139 webserver.cc:460] Webserver started at http://127.0.0.1:41309/ using document root <none> and password file <none>
I20260806 02:16:40.419772 8139 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:40.419866 8139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:40.420105 8139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:40.422197 8139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "efce7231290b4195925414054f05d62a"
format_stamp: "Formatted at 2026-08-06 02:16:40 on dist-test-slave-6k22"
server_key: "ef5a1fe04afb2547fd0c11c515aa2c48"
server_key_iv: "c4fe3ad7b962b356944cc9f79630e786"
server_key_version: "encryptionkey@0"
I20260806 02:16:40.422814 8139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "efce7231290b4195925414054f05d62a"
format_stamp: "Formatted at 2026-08-06 02:16:40 on dist-test-slave-6k22"
server_key: "ef5a1fe04afb2547fd0c11c515aa2c48"
server_key_iv: "c4fe3ad7b962b356944cc9f79630e786"
server_key_version: "encryptionkey@0"
I20260806 02:16:40.426458 8139 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 02:16:40.428894 8154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:40.430107 8139 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 02:16:40.430275 8139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "efce7231290b4195925414054f05d62a"
format_stamp: "Formatted at 2026-08-06 02:16:40 on dist-test-slave-6k22"
server_key: "ef5a1fe04afb2547fd0c11c515aa2c48"
server_key_iv: "c4fe3ad7b962b356944cc9f79630e786"
server_key_version: "encryptionkey@0"
I20260806 02:16:40.430385 8139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:40.453722 8139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:40.456460 8139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:40.456648 8139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:40.463682 8139 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34589
I20260806 02:16:40.463725 8212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34589 every 8 connection(s)
I20260806 02:16:40.464598 8139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:40.467235 8213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:40.468976 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 8139
I20260806 02:16:40.469121 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:40.469406 18713 external_mini_cluster.cc:1428] Setting key c57035ca60d10f6dd7263bef3f800662
I20260806 02:16:40.473955 8213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a: Bootstrap starting.
I20260806 02:16:40.476507 8213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:40.477226 8213 log.cc:824] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:40.479269 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:40.470817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50461 (local address 127.0.0.1:34589)
0806 02:16:40.471229 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:40.471249 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:40.471292 (+ 43us) server_negotiation.cc:407] Connection header received
0806 02:16:40.472144 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:40.472170 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:40.472586 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:40.472958 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:40.473373 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:40.474750 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:40.475681 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:40.476356 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:40.476526 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:40.477355 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:40.477363 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:40.477380 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:40.477841 (+ 461us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:40.477891 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:40.477939 (+ 48us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:40.478051 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:40.478170 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 02:16:40.478375 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":111,"threads_started":1}
I20260806 02:16:40.479378 8213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a: No bootstrap required, opened a new log
I20260806 02:16:40.482271 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:40.483232 8213 raft_consensus.cc:348] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efce7231290b4195925414054f05d62a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34589 } }
I20260806 02:16:40.483472 8213 raft_consensus.cc:374] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:40.483587 8213 raft_consensus.cc:729] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efce7231290b4195925414054f05d62a, State: Initialized, Role: FOLLOWER
I20260806 02:16:40.484077 8213 consensus_queue.cc:260] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efce7231290b4195925414054f05d62a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34589 } }
I20260806 02:16:40.484197 8213 raft_consensus.cc:388] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:40.484280 8213 raft_consensus.cc:482] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:40.484395 8213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:40.485294 8213 raft_consensus.cc:504] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efce7231290b4195925414054f05d62a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34589 } }
I20260806 02:16:40.485627 8213 leader_election.cc:302] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efce7231290b4195925414054f05d62a; no voters:
I20260806 02:16:40.485980 8213 leader_election.cc:288] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:40.486045 8223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:40.486274 8223 raft_consensus.cc:686] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [term 1 LEADER]: Becoming Leader. State: Replica: efce7231290b4195925414054f05d62a, State: Running, Role: LEADER
I20260806 02:16:40.486613 8223 consensus_queue.cc:237] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efce7231290b4195925414054f05d62a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34589 } }
I20260806 02:16:40.487205 8213 sys_catalog.cc:564] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:40.487757 8223 sys_catalog.cc:455] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [sys.catalog]: SysCatalogTable state changed. Reason: New leader efce7231290b4195925414054f05d62a. Latest consensus state: current_term: 1 leader_uuid: "efce7231290b4195925414054f05d62a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efce7231290b4195925414054f05d62a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34589 } } }
I20260806 02:16:40.487943 8223 sys_catalog.cc:458] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:40.488368 8230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:40.488639 8224 sys_catalog.cc:455] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efce7231290b4195925414054f05d62a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efce7231290b4195925414054f05d62a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34589 } } }
I20260806 02:16:40.488824 8224 sys_catalog.cc:458] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:40.489519 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:40.484543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55277 (local address 127.0.0.1:34589)
0806 02:16:40.484676 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:40.484681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:40.484944 (+ 263us) server_negotiation.cc:407] Connection header received
0806 02:16:40.485118 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:40.485122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:40.485176 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:40.485276 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:40.485598 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:40.486336 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:40.487424 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:40.487840 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:40.487980 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:40.489016 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:40.489021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:40.489025 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:40.489040 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:40.489071 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:40.489090 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:40.489188 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:40.489307 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 02:16:40.489362 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 02:16:40.491806 8230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:40.497996 8230 catalog_manager.cc:1347] Generated new cluster ID: 230aeea218a544a8840371a32545276f
I20260806 02:16:40.498085 8230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:40.524688 8230 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:40.525817 8230 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:40.532624 8230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efce7231290b4195925414054f05d62a: Generated new TSK 0
I20260806 02:16:40.533316 8230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:40.542711 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 8139
2026-08-06T02:16:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:40 dist-test-slave-6k22 krb5kdc[8246](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:40 dist-test-slave-6k22 krb5kdc[8246](info): set up 2 sockets
Aug 06 02:16:40 dist-test-slave-6k22 krb5kdc[8246](info): commencing operation
krb5kdc: starting...
W20260806 02:16:41.592903 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:41 dist-test-slave-6k22 krb5kdc[8246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 02:16:41.715958 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42785
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:41.825691 8260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:41.825991 8260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:41.826069 8260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:41.830109 8260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:41.830188 8260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:41.830224 8260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:41.830260 8260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:41.830297 8260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:41.835410 8260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:41.836485 8260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:41.837502 8260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:41.845773 8265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:41.845815 8266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:41.845773 8268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:41.846094 8260 server_base.cc:1034] running on GCE node
I20260806 02:16:41.846642 8260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:41.847785 8260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:41.848973 8260 hybrid_clock.cc:648] HybridClock initialized: now 1785982601848953 us; error 30 us; skew 500 ppm
Aug 06 02:16:41 dist-test-slave-6k22 krb5kdc[8246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982601, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:41.851773 8260 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 02:16:41.852677 8260 webserver.cc:460] Webserver started at http://127.0.0.1:34351/ using document root <none> and password file <none>
I20260806 02:16:41.853235 8260 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:41.853343 8260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:41.853582 8260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:41.855614 8260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "5318577c2dfd46c88f659cab189440b9"
format_stamp: "Formatted at 2026-08-06 02:16:41 on dist-test-slave-6k22"
server_key: "1e65bf7c776a7f776a1eaa8b22255cc4"
server_key_iv: "6a1ef518943ce1f2d52069ad81299a35"
server_key_version: "encryptionkey@0"
I20260806 02:16:41.856191 8260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "5318577c2dfd46c88f659cab189440b9"
format_stamp: "Formatted at 2026-08-06 02:16:41 on dist-test-slave-6k22"
server_key: "1e65bf7c776a7f776a1eaa8b22255cc4"
server_key_iv: "6a1ef518943ce1f2d52069ad81299a35"
server_key_version: "encryptionkey@0"
I20260806 02:16:41.859797 8260 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 02:16:41.862028 8276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:41.863140 8260 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:16:41.863289 8260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "5318577c2dfd46c88f659cab189440b9"
format_stamp: "Formatted at 2026-08-06 02:16:41 on dist-test-slave-6k22"
server_key: "1e65bf7c776a7f776a1eaa8b22255cc4"
server_key_iv: "6a1ef518943ce1f2d52069ad81299a35"
server_key_version: "encryptionkey@0"
I20260806 02:16:41.863418 8260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:41.867949 8260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:41.871086 8260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:41.871295 8260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:41.878423 8260 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42785
I20260806 02:16:41.878453 8333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42785 every 8 connection(s)
I20260806 02:16:41.879304 8260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:41.881666 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 8260
I20260806 02:16:41.881819 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:41.882148 18713 external_mini_cluster.cc:1428] Setting key 344f95565d40555d403480a1080f76ee
I20260806 02:16:41.882268 8334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:41.887283 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:41.883513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45247 (local address 127.0.0.1:42785)
0806 02:16:41.883978 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:41.883992 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:41.884026 (+ 34us) server_negotiation.cc:407] Connection header received
0806 02:16:41.884909 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:41.884928 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:41.885199 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:41.885519 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:41.886172 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:41.886177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:41.886194 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:41.886208 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:41.886254 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:41.886271 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:41.886403 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:41.886504 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 02:16:41.886647 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":139,"threads_started":1}
I20260806 02:16:41.888777 8334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9: Bootstrap starting.
I20260806 02:16:41.890012 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:41.891202 8334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:41.892017 8334 log.cc:824] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:41.893805 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:41.892041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37769 (local address 127.0.0.1:42785)
0806 02:16:41.892165 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:41.892169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:41.892584 (+ 415us) server_negotiation.cc:407] Connection header received
0806 02:16:41.892663 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:41.892668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:41.892700 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:41.892782 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:41.893420 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:41.893422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:41.893424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:41.893434 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:41.893455 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:41.893464 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:41.893528 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:41.893640 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 02:16:41.893667 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 02:16:41.894222 8334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9: No bootstrap required, opened a new log
I20260806 02:16:41.897544 8334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5318577c2dfd46c88f659cab189440b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42785 } }
I20260806 02:16:41.897773 8334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:41.897842 8334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5318577c2dfd46c88f659cab189440b9, State: Initialized, Role: FOLLOWER
I20260806 02:16:41.898239 8334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5318577c2dfd46c88f659cab189440b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42785 } }
I20260806 02:16:41.898396 8334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:41.898470 8334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:41.898583 8334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:41.899343 8334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5318577c2dfd46c88f659cab189440b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42785 } }
I20260806 02:16:41.899686 8334 leader_election.cc:302] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5318577c2dfd46c88f659cab189440b9; no voters:
I20260806 02:16:41.899936 8334 leader_election.cc:288] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:41.900048 8346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:41.900276 8346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [term 1 LEADER]: Becoming Leader. State: Replica: 5318577c2dfd46c88f659cab189440b9, State: Running, Role: LEADER
I20260806 02:16:41.900605 8346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5318577c2dfd46c88f659cab189440b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42785 } }
I20260806 02:16:41.901117 8334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:41.902381 8347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5318577c2dfd46c88f659cab189440b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5318577c2dfd46c88f659cab189440b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42785 } } }
I20260806 02:16:41.902519 8347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:41.902905 8355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:41.902876 8348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5318577c2dfd46c88f659cab189440b9. Latest consensus state: current_term: 1 leader_uuid: "5318577c2dfd46c88f659cab189440b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5318577c2dfd46c88f659cab189440b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42785 } } }
I20260806 02:16:41.903021 8348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:41.905805 8355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:41.911019 8355 catalog_manager.cc:1347] Generated new cluster ID: aedf3b9eef7b4162adf3c0a2084b616f
I20260806 02:16:41.911082 8355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:41.926389 8355 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:41.926680 8355 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:41.934460 8355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5318577c2dfd46c88f659cab189440b9: Generated new TSK 0
I20260806 02:16:41.935130 8355 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:41.941823 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 8260
2026-08-06T02:16:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:42 dist-test-slave-6k22 krb5kdc[8367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:42 dist-test-slave-6k22 krb5kdc[8367](info): set up 2 sockets
Aug 06 02:16:42 dist-test-slave-6k22 krb5kdc[8367](info): commencing operation
krb5kdc: starting...
W20260806 02:16:43.009485 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:43 dist-test-slave-6k22 krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 02:16:43.132745 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46181
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:43.246428 8380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:43.246672 8380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:43.246711 8380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:43.250593 8380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:43.250655 8380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:43.250679 8380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:43.250697 8380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:43.250715 8380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:43.255589 8380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:43.256495 8380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:43.257541 8380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:43.265990 8386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:43.265997 8390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:43.266124 8388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:43.266227 8380 server_base.cc:1034] running on GCE node
I20260806 02:16:43.266839 8380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:43.267890 8380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:43.269074 8380 hybrid_clock.cc:648] HybridClock initialized: now 1785982603269050 us; error 33 us; skew 500 ppm
Aug 06 02:16:43 dist-test-slave-6k22 krb5kdc[8367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982603, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:43.276815 8380 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 02:16:43.277793 8380 webserver.cc:460] Webserver started at http://127.0.0.1:38047/ using document root <none> and password file <none>
I20260806 02:16:43.278400 8380 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:43.278493 8380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:43.278735 8380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:43.280822 8380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "dc4b82fbe850449ba2c7bddaadb8c562"
format_stamp: "Formatted at 2026-08-06 02:16:43 on dist-test-slave-6k22"
server_key: "db2deefb13feed3b69768051e37bbdee"
server_key_iv: "3f166be79057344515d1027374402766"
server_key_version: "encryptionkey@0"
I20260806 02:16:43.281392 8380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "dc4b82fbe850449ba2c7bddaadb8c562"
format_stamp: "Formatted at 2026-08-06 02:16:43 on dist-test-slave-6k22"
server_key: "db2deefb13feed3b69768051e37bbdee"
server_key_iv: "3f166be79057344515d1027374402766"
server_key_version: "encryptionkey@0"
I20260806 02:16:43.285038 8380 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:16:43.287315 8398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:43.288540 8380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:43.288694 8380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "dc4b82fbe850449ba2c7bddaadb8c562"
format_stamp: "Formatted at 2026-08-06 02:16:43 on dist-test-slave-6k22"
server_key: "db2deefb13feed3b69768051e37bbdee"
server_key_iv: "3f166be79057344515d1027374402766"
server_key_version: "encryptionkey@0"
I20260806 02:16:43.288805 8380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:43.306994 8380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:43.309969 8380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:43.310159 8380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:43.317198 8380 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46181
I20260806 02:16:43.317220 8451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46181 every 8 connection(s)
I20260806 02:16:43.318161 8380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:43.319157 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 8380
I20260806 02:16:43.319309 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:43.319628 18713 external_mini_cluster.cc:1428] Setting key f107c4d139d4c711435caa7bc95197c4
I20260806 02:16:43.321223 8452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:43.326481 8452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562: Bootstrap starting.
Aug 06 02:16:43 dist-test-slave-6k22 krb5kdc[8367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 02:16:43.328866 8452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:43.329763 8452 log.cc:824] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:43.331681 8452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562: No bootstrap required, opened a new log
I20260806 02:16:43.333581 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:43.320965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:57545 (local address 127.0.0.1:46181)
0806 02:16:43.321475 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:43.321490 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:43.321524 (+ 34us) server_negotiation.cc:407] Connection header received
0806 02:16:43.322172 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:43.322188 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:43.322505 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:43.322834 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:43.323717 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:43.324654 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:43.325512 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:43.325963 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:43.326123 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:43.328755 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:43.328761 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:43.328773 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:43.328799 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:43.330642 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:43.331159 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:43.331169 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:43.331173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:43.331282 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:43.331688 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:43.331706 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:43.331707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:43.332099 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:43.332330 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:43.332664 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 02:16:43.332925 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":148,"threads_started":1}
I20260806 02:16:43.335446 8452 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4b82fbe850449ba2c7bddaadb8c562" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46181 } }
I20260806 02:16:43.335726 8452 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:43.335803 8452 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc4b82fbe850449ba2c7bddaadb8c562, State: Initialized, Role: FOLLOWER
I20260806 02:16:43.336288 8452 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4b82fbe850449ba2c7bddaadb8c562" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46181 } }
I20260806 02:16:43.336469 8452 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:43.336541 8452 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:43.336647 8452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:43.336740 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:43.337472 8452 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4b82fbe850449ba2c7bddaadb8c562" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46181 } }
I20260806 02:16:43.337818 8452 leader_election.cc:302] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc4b82fbe850449ba2c7bddaadb8c562; no voters:
I20260806 02:16:43.338178 8452 leader_election.cc:288] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:43.338241 8461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:43.338485 8461 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [term 1 LEADER]: Becoming Leader. State: Replica: dc4b82fbe850449ba2c7bddaadb8c562, State: Running, Role: LEADER
I20260806 02:16:43.338814 8461 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4b82fbe850449ba2c7bddaadb8c562" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46181 } }
I20260806 02:16:43.339421 8452 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:43.340039 8461 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc4b82fbe850449ba2c7bddaadb8c562. Latest consensus state: current_term: 1 leader_uuid: "dc4b82fbe850449ba2c7bddaadb8c562" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4b82fbe850449ba2c7bddaadb8c562" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46181 } } }
I20260806 02:16:43.340171 8461 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:43.340466 8464 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc4b82fbe850449ba2c7bddaadb8c562" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc4b82fbe850449ba2c7bddaadb8c562" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46181 } } }
I20260806 02:16:43.340601 8464 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:43.340581 8472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:43.344266 8472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:43.348481 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:43.338972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:40191 (local address 127.0.0.1:46181)
0806 02:16:43.339057 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:43.339061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:43.339249 (+ 188us) server_negotiation.cc:407] Connection header received
0806 02:16:43.339392 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:43.339395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:43.339435 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:43.339526 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:43.340542 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:43.341205 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:43.342444 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:43.342820 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:43.342910 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:43.344636 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:43.344640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:43.344642 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:43.344666 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:43.346247 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:43.346850 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:43.346854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:43.346856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:43.346940 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:43.347365 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:43.347368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:43.347370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:43.347594 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:43.347721 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:43.348161 (+ 440us) server_negotiation.cc:299] Negotiation successful
0806 02:16:43.348271 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 02:16:43.350045 8472 catalog_manager.cc:1347] Generated new cluster ID: 2979f461b34442c38b17530c5f0fe344
I20260806 02:16:43.350102 8472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:43.373277 8472 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:43.374282 8472 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:43.378355 8472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc4b82fbe850449ba2c7bddaadb8c562: Generated new TSK 0
I20260806 02:16:43.379007 8472 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:43.384909 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 8380
W20260806 02:16:43.392798 8365 connection.cc:535] client connection to 127.0.0.1:46181 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46181): Connection reset by peer (error 104)
2026-08-06T02:16:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:43 dist-test-slave-6k22 krb5kdc[8486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:43 dist-test-slave-6k22 krb5kdc[8486](info): set up 2 sockets
Aug 06 02:16:43 dist-test-slave-6k22 krb5kdc[8486](info): commencing operation
krb5kdc: starting...
W20260806 02:16:44.441010 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:44 dist-test-slave-6k22 krb5kdc[8486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 02:16:44.565873 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42907
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:44.676920 8502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:44.677182 8502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:44.677223 8502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:44.681034 8502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:44.681082 8502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:44.681102 8502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:44.681118 8502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:44.681133 8502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:44.686013 8502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:44.686908 8502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:44.687886 8502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:44.696097 8508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:44.696136 8507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:44.696177 8502 server_base.cc:1034] running on GCE node
W20260806 02:16:44.696096 8510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:44.696879 8502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:44.697875 8502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:44.699034 8502 hybrid_clock.cc:648] HybridClock initialized: now 1785982604699016 us; error 27 us; skew 500 ppm
Aug 06 02:16:44 dist-test-slave-6k22 krb5kdc[8486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982604, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:44.701720 8502 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 02:16:44.702592 8502 webserver.cc:460] Webserver started at http://127.0.0.1:38477/ using document root <none> and password file <none>
I20260806 02:16:44.703150 8502 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:44.703207 8502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:44.703433 8502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:44.705389 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "00e8c7f4426d406fbe613b31221a2d2b"
format_stamp: "Formatted at 2026-08-06 02:16:44 on dist-test-slave-6k22"
server_key: "247519846254a6e57c9276c9f1b5ab4d"
server_key_iv: "c73d6d76cd431d1e59678699753045ba"
server_key_version: "encryptionkey@0"
I20260806 02:16:44.705933 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "00e8c7f4426d406fbe613b31221a2d2b"
format_stamp: "Formatted at 2026-08-06 02:16:44 on dist-test-slave-6k22"
server_key: "247519846254a6e57c9276c9f1b5ab4d"
server_key_iv: "c73d6d76cd431d1e59678699753045ba"
server_key_version: "encryptionkey@0"
I20260806 02:16:44.709627 8502 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:16:44.712052 8518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:44.713301 8502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:44.713442 8502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "00e8c7f4426d406fbe613b31221a2d2b"
format_stamp: "Formatted at 2026-08-06 02:16:44 on dist-test-slave-6k22"
server_key: "247519846254a6e57c9276c9f1b5ab4d"
server_key_iv: "c73d6d76cd431d1e59678699753045ba"
server_key_version: "encryptionkey@0"
I20260806 02:16:44.713552 8502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:44.742925 8502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:44.746198 8502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:44.746414 8502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:44.754033 8502 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42907
I20260806 02:16:44.754057 8574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42907 every 8 connection(s)
I20260806 02:16:44.755010 8502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:44.757792 8575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:44.762053 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 8502
I20260806 02:16:44.762176 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:44.762449 18713 external_mini_cluster.cc:1428] Setting key 0e5f33ae487e8ccf56b85ce3db9f8167
I20260806 02:16:44.763803 8575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b: Bootstrap starting.
I20260806 02:16:44.766036 8575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:44.766741 8575 log.cc:824] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:44.768553 8575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b: No bootstrap required, opened a new log
I20260806 02:16:44.771289 8578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:44.763923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:51001 (local address 127.0.0.1:42907)
0806 02:16:44.764340 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:44.764355 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:44.764389 (+ 34us) server_negotiation.cc:407] Connection header received
0806 02:16:44.765060 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:44.765077 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:44.765420 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:44.765754 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:44.766212 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:44.767194 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:44.768129 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:44.768580 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:44.768759 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:44.769638 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:44.769643 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:44.769656 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:44.770065 (+ 409us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:44.770118 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:44.770141 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:44.770292 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:44.770477 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 02:16:44.770608 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":130,"threads_started":1}
I20260806 02:16:44.772071 8575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e8c7f4426d406fbe613b31221a2d2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42907 } }
I20260806 02:16:44.772275 8575 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:44.772342 8575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00e8c7f4426d406fbe613b31221a2d2b, State: Initialized, Role: FOLLOWER
I20260806 02:16:44.772789 8575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e8c7f4426d406fbe613b31221a2d2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42907 } }
I20260806 02:16:44.772935 8575 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:44.773025 8575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:44.773119 8575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:44.774047 8575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e8c7f4426d406fbe613b31221a2d2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42907 } }
I20260806 02:16:44.774115 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:44.774407 8575 leader_election.cc:302] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00e8c7f4426d406fbe613b31221a2d2b; no voters:
I20260806 02:16:44.774700 8575 leader_election.cc:288] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:44.774766 8580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:44.774950 8580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [term 1 LEADER]: Becoming Leader. State: Replica: 00e8c7f4426d406fbe613b31221a2d2b, State: Running, Role: LEADER
I20260806 02:16:44.775315 8580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e8c7f4426d406fbe613b31221a2d2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42907 } }
I20260806 02:16:44.775964 8575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:44.777130 8581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00e8c7f4426d406fbe613b31221a2d2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e8c7f4426d406fbe613b31221a2d2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42907 } } }
I20260806 02:16:44.777141 8582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00e8c7f4426d406fbe613b31221a2d2b. Latest consensus state: current_term: 1 leader_uuid: "00e8c7f4426d406fbe613b31221a2d2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e8c7f4426d406fbe613b31221a2d2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42907 } } }
I20260806 02:16:44.777395 8582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:44.777695 8581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:44.778029 8596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:44.781090 8596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:44.781289 8578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:44.776311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:45009 (local address 127.0.0.1:42907)
0806 02:16:44.776404 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:44.776407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:44.776828 (+ 421us) server_negotiation.cc:407] Connection header received
0806 02:16:44.777039 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:44.777044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:44.777099 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:44.777163 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:44.777604 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:44.778350 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:44.779515 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:44.779818 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:44.779929 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:44.780766 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:44.780775 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:44.780777 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:44.780787 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:44.780810 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:44.780820 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:44.780918 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:44.781041 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 02:16:44.781072 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 02:16:44.786692 8596 catalog_manager.cc:1347] Generated new cluster ID: 0f89cefb4ee548178e5f8a0cb99c340e
I20260806 02:16:44.786761 8596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:44.807269 8596 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:44.808174 8596 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:44.816869 8596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00e8c7f4426d406fbe613b31221a2d2b: Generated new TSK 0
I20260806 02:16:44.817744 8596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:44.825641 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 8502
W20260806 02:16:44.834090 8484 connection.cc:535] client connection to 127.0.0.1:42907 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42907): Connection reset by peer (error 104)
2026-08-06T02:16:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:44 dist-test-slave-6k22 krb5kdc[8606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:44 dist-test-slave-6k22 krb5kdc[8606](info): set up 2 sockets
Aug 06 02:16:44 dist-test-slave-6k22 krb5kdc[8606](info): commencing operation
krb5kdc: starting...
W20260806 02:16:45.889017 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:45 dist-test-slave-6k22 krb5kdc[8606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 02:16:46.026932 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40131
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:46.135497 8620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:46.135731 8620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:46.135769 8620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:46.139727 8620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:46.139775 8620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:46.139811 8620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:46.139863 8620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:46.139894 8620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:46.144876 8620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:46.145798 8620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:46.146761 8620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:46.155045 8626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:46.155048 8625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:46.155129 8628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:46.155424 8620 server_base.cc:1034] running on GCE node
I20260806 02:16:46.155992 8620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:46.156968 8620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:46.158139 8620 hybrid_clock.cc:648] HybridClock initialized: now 1785982606158114 us; error 37 us; skew 500 ppm
Aug 06 02:16:46 dist-test-slave-6k22 krb5kdc[8606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982606, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:46.160802 8620 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 02:16:46.161684 8620 webserver.cc:460] Webserver started at http://127.0.0.1:36201/ using document root <none> and password file <none>
I20260806 02:16:46.162212 8620 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:46.162271 8620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:46.162519 8620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:46.164518 8620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "d6161394432149b987b4323d2925ec9a"
format_stamp: "Formatted at 2026-08-06 02:16:46 on dist-test-slave-6k22"
server_key: "a31c6c5b43f5ec3bcdcb72d640e77568"
server_key_iv: "05b0010fa2ebbb074a75b64f01cad342"
server_key_version: "encryptionkey@0"
I20260806 02:16:46.165025 8620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "d6161394432149b987b4323d2925ec9a"
format_stamp: "Formatted at 2026-08-06 02:16:46 on dist-test-slave-6k22"
server_key: "a31c6c5b43f5ec3bcdcb72d640e77568"
server_key_iv: "05b0010fa2ebbb074a75b64f01cad342"
server_key_version: "encryptionkey@0"
I20260806 02:16:46.168717 8620 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 02:16:46.171007 8635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:46.172250 8620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 02:16:46.172405 8620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "d6161394432149b987b4323d2925ec9a"
format_stamp: "Formatted at 2026-08-06 02:16:46 on dist-test-slave-6k22"
server_key: "a31c6c5b43f5ec3bcdcb72d640e77568"
server_key_iv: "05b0010fa2ebbb074a75b64f01cad342"
server_key_version: "encryptionkey@0"
I20260806 02:16:46.172565 8620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:46.176858 8620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:46.180869 8620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:46.181067 8620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:46.187674 8620 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40131
I20260806 02:16:46.187700 8695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40131 every 8 connection(s)
I20260806 02:16:46.188501 8620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:46.191130 8696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:46.192420 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 8620
I20260806 02:16:46.192546 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:46.192834 18713 external_mini_cluster.cc:1428] Setting key 8936467169dfc611e7e158fc6acd5f42
I20260806 02:16:46.197609 8696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a: Bootstrap starting.
I20260806 02:16:46.198202 8699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:46.194187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:39977 (local address 127.0.0.1:40131)
0806 02:16:46.194598 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:46.194617 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:46.194657 (+ 40us) server_negotiation.cc:407] Connection header received
0806 02:16:46.195481 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:46.195505 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:46.195890 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:46.196229 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:46.196886 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:46.196893 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:46.196907 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:46.196919 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:46.196977 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:46.197003 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:46.197134 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:46.197199 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 02:16:46.197350 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":97,"threads_started":1}
I20260806 02:16:46.200141 8696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:46.201035 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:46.201020 8696 log.cc:824] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:46.203037 8696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a: No bootstrap required, opened a new log
I20260806 02:16:46.204770 8699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:46.203059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:44747 (local address 127.0.0.1:40131)
0806 02:16:46.203180 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:46.203186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:46.203388 (+ 202us) server_negotiation.cc:407] Connection header received
0806 02:16:46.203513 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:46.203516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:46.203550 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:46.203674 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:46.204400 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:46.204403 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:46.204406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:46.204417 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:46.204443 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:46.204461 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:46.204548 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:46.204622 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 02:16:46.204660 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 02:16:46.207058 8696 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6161394432149b987b4323d2925ec9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40131 } }
I20260806 02:16:46.207252 8696 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:46.207324 8696 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6161394432149b987b4323d2925ec9a, State: Initialized, Role: FOLLOWER
I20260806 02:16:46.207773 8696 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6161394432149b987b4323d2925ec9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40131 } }
I20260806 02:16:46.207895 8696 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:46.207975 8696 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:46.208109 8696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:46.208950 8696 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6161394432149b987b4323d2925ec9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40131 } }
I20260806 02:16:46.209280 8696 leader_election.cc:302] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6161394432149b987b4323d2925ec9a; no voters:
I20260806 02:16:46.209575 8696 leader_election.cc:288] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:46.209736 8706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:46.209987 8706 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [term 1 LEADER]: Becoming Leader. State: Replica: d6161394432149b987b4323d2925ec9a, State: Running, Role: LEADER
I20260806 02:16:46.210340 8706 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6161394432149b987b4323d2925ec9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40131 } }
I20260806 02:16:46.210841 8696 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:46.211948 8708 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6161394432149b987b4323d2925ec9a. Latest consensus state: current_term: 1 leader_uuid: "d6161394432149b987b4323d2925ec9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6161394432149b987b4323d2925ec9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40131 } } }
I20260806 02:16:46.212109 8708 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:46.212558 8715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:46.212527 8707 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6161394432149b987b4323d2925ec9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6161394432149b987b4323d2925ec9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40131 } } }
I20260806 02:16:46.212739 8707 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:46.215176 8715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:46.220943 8715 catalog_manager.cc:1347] Generated new cluster ID: b984cae87d9a4a86992eb77d890879ee
I20260806 02:16:46.221004 8715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:46.236255 8715 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:46.236580 8715 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:46.244825 8715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6161394432149b987b4323d2925ec9a: Generated new TSK 0
I20260806 02:16:46.245625 8715 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:46.251109 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 8620
2026-08-06T02:16:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:46 dist-test-slave-6k22 krb5kdc[8728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:46 dist-test-slave-6k22 krb5kdc[8728](info): set up 2 sockets
Aug 06 02:16:46 dist-test-slave-6k22 krb5kdc[8728](info): commencing operation
krb5kdc: starting...
W20260806 02:16:47.306048 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:47 dist-test-slave-6k22 krb5kdc[8728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 02:16:47.432475 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37133
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:47.541989 8741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:47.542255 8741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:47.542327 8741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:47.546299 8741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:47.546351 8741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:47.546371 8741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:47.546387 8741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:47.546442 8741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:47.551429 8741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37133
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:47.552431 8741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:47.553328 8741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:47.561612 8748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:47.561749 8741 server_base.cc:1034] running on GCE node
W20260806 02:16:47.561635 8747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:47.561743 8750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:47.562480 8741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:47.563542 8741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:47.564774 8741 hybrid_clock.cc:648] HybridClock initialized: now 1785982607564760 us; error 33 us; skew 500 ppm
Aug 06 02:16:47 dist-test-slave-6k22 krb5kdc[8728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982607, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:47.567519 8741 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 02:16:47.568470 8741 webserver.cc:460] Webserver started at http://127.0.0.1:37673/ using document root <none> and password file <none>
I20260806 02:16:47.569022 8741 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:47.569111 8741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:47.569348 8741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:47.571435 8741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "022564f38d544d03b479ec83b5db1248"
format_stamp: "Formatted at 2026-08-06 02:16:47 on dist-test-slave-6k22"
server_key: "4c18e334ae47bfc884c88731a671d388"
server_key_iv: "1a4427b5a8adae9fa9d43fc494c101e2"
server_key_version: "encryptionkey@0"
I20260806 02:16:47.572125 8741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "022564f38d544d03b479ec83b5db1248"
format_stamp: "Formatted at 2026-08-06 02:16:47 on dist-test-slave-6k22"
server_key: "4c18e334ae47bfc884c88731a671d388"
server_key_iv: "1a4427b5a8adae9fa9d43fc494c101e2"
server_key_version: "encryptionkey@0"
I20260806 02:16:47.575868 8741 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 02:16:47.578382 8758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:47.579732 8741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:47.579871 8741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "022564f38d544d03b479ec83b5db1248"
format_stamp: "Formatted at 2026-08-06 02:16:47 on dist-test-slave-6k22"
server_key: "4c18e334ae47bfc884c88731a671d388"
server_key_iv: "1a4427b5a8adae9fa9d43fc494c101e2"
server_key_version: "encryptionkey@0"
I20260806 02:16:47.579989 8741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:47.595727 8741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:47.598961 8741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:47.599162 8741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:47.606213 8741 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37133
I20260806 02:16:47.606243 8814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37133 every 8 connection(s)
I20260806 02:16:47.607144 8741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:47.608446 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 8741
I20260806 02:16:47.608624 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:47.608954 18713 external_mini_cluster.cc:1428] Setting key 6632c91e846d95e2aee2ad1b8c5bf9a2
I20260806 02:16:47.610423 8815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:47.616228 8815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248: Bootstrap starting.
Aug 06 02:16:47 dist-test-slave-6k22 krb5kdc[8728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 02:16:47.618647 8815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:47.619532 8815 log.cc:824] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:47.621469 8815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248: No bootstrap required, opened a new log
I20260806 02:16:47.623277 8818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:47.610317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:37171 (local address 127.0.0.1:37133)
0806 02:16:47.610744 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:47.610759 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:47.610794 (+ 35us) server_negotiation.cc:407] Connection header received
0806 02:16:47.611458 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:47.611474 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:47.611937 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:47.612268 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:47.613108 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:47.614155 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:47.615010 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:47.615483 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:47.615676 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:47.618457 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:47.618462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:47.618474 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:47.618500 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:47.620464 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:47.620963 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:47.620973 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:47.620978 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:47.621081 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:47.621443 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:47.621446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:47.621448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:47.621914 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:47.622138 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:47.622426 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 02:16:47.622631 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":138,"threads_started":1}
I20260806 02:16:47.625162 8815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "022564f38d544d03b479ec83b5db1248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37133 } }
I20260806 02:16:47.625435 8815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:47.625497 8815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 022564f38d544d03b479ec83b5db1248, State: Initialized, Role: FOLLOWER
I20260806 02:16:47.625948 8815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "022564f38d544d03b479ec83b5db1248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37133 } }
I20260806 02:16:47.626116 8815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:47.626257 8815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:47.626300 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:47.626374 8815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:47.627261 8815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "022564f38d544d03b479ec83b5db1248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37133 } }
I20260806 02:16:47.627660 8815 leader_election.cc:302] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 022564f38d544d03b479ec83b5db1248; no voters:
I20260806 02:16:47.627983 8815 leader_election.cc:288] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:47.628058 8824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:47.628260 8824 raft_consensus.cc:686] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [term 1 LEADER]: Becoming Leader. State: Replica: 022564f38d544d03b479ec83b5db1248, State: Running, Role: LEADER
I20260806 02:16:47.628576 8824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "022564f38d544d03b479ec83b5db1248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37133 } }
I20260806 02:16:47.629177 8815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:47.630524 8825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "022564f38d544d03b479ec83b5db1248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "022564f38d544d03b479ec83b5db1248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37133 } } }
I20260806 02:16:47.630651 8825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:47.630959 8826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 022564f38d544d03b479ec83b5db1248. Latest consensus state: current_term: 1 leader_uuid: "022564f38d544d03b479ec83b5db1248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "022564f38d544d03b479ec83b5db1248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37133 } } }
I20260806 02:16:47.631075 8826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:47.632534 8836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:47.635782 8836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:47.637923 8818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:47.628450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:56157 (local address 127.0.0.1:37133)
0806 02:16:47.628533 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:47.628537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:47.629051 (+ 514us) server_negotiation.cc:407] Connection header received
0806 02:16:47.629160 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:47.629164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:47.629201 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:47.629290 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:47.630146 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:47.631062 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:47.632387 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:47.632624 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:47.632724 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:47.634265 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:47.634270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:47.634273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:47.634298 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:47.636023 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:47.636553 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:47.636556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:47.636558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:47.636643 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:47.637034 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:47.637038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:47.637039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:47.637237 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:47.637392 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:47.637685 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 02:16:47.637776 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 02:16:47.641551 8836 catalog_manager.cc:1347] Generated new cluster ID: ab2c78e37d644dd0921510475f60a5a2
I20260806 02:16:47.641611 8836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:47.658039 8836 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:47.659245 8836 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:47.667928 8836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 022564f38d544d03b479ec83b5db1248: Generated new TSK 0
I20260806 02:16:47.668678 8836 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:47.674479 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 8741
W20260806 02:16:47.682420 8726 connection.cc:535] client connection to 127.0.0.1:37133 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37133): Connection reset by peer (error 104)
2026-08-06T02:16:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:47 dist-test-slave-6k22 krb5kdc[8848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:47 dist-test-slave-6k22 krb5kdc[8848](info): set up 2 sockets
Aug 06 02:16:47 dist-test-slave-6k22 krb5kdc[8848](info): commencing operation
krb5kdc: starting...
W20260806 02:16:48.737404 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:48 dist-test-slave-6k22 krb5kdc[8848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 02:16:48.863476 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34487
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:48.974440 8861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:48.974689 8861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:48.974730 8861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:48.978734 8861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:48.978789 8861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:48.978808 8861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:48.978854 8861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:48.978888 8861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:48.984091 8861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34487
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:48.985146 8861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:48.986169 8861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:48.994781 8867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:48.994781 8866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:48.995067 8861 server_base.cc:1034] running on GCE node
W20260806 02:16:48.995143 8870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:48.995816 8861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:48.996861 8861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:48.998029 8861 hybrid_clock.cc:648] HybridClock initialized: now 1785982608998010 us; error 32 us; skew 500 ppm
Aug 06 02:16:49 dist-test-slave-6k22 krb5kdc[8848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982609, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:49.007296 8861 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 02:16:49.008333 8861 webserver.cc:460] Webserver started at http://127.0.0.1:38279/ using document root <none> and password file <none>
I20260806 02:16:49.008937 8861 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:49.009032 8861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:49.009274 8861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:49.011363 8861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "fdc0f73a598f4846976b2293c8374dfb"
format_stamp: "Formatted at 2026-08-06 02:16:49 on dist-test-slave-6k22"
server_key: "c5bb119eb15c6a52b0d45eadaf1b54c5"
server_key_iv: "7589c1c78ee0d00457c99607599a06e5"
server_key_version: "encryptionkey@0"
I20260806 02:16:49.011991 8861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "fdc0f73a598f4846976b2293c8374dfb"
format_stamp: "Formatted at 2026-08-06 02:16:49 on dist-test-slave-6k22"
server_key: "c5bb119eb15c6a52b0d45eadaf1b54c5"
server_key_iv: "7589c1c78ee0d00457c99607599a06e5"
server_key_version: "encryptionkey@0"
I20260806 02:16:49.015686 8861 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:16:49.018083 8878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:49.019357 8861 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:16:49.019505 8861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "fdc0f73a598f4846976b2293c8374dfb"
format_stamp: "Formatted at 2026-08-06 02:16:49 on dist-test-slave-6k22"
server_key: "c5bb119eb15c6a52b0d45eadaf1b54c5"
server_key_iv: "7589c1c78ee0d00457c99607599a06e5"
server_key_version: "encryptionkey@0"
I20260806 02:16:49.019665 8861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:49.037765 8861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:49.040618 8861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:49.040829 8861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:49.048553 8861 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34487
I20260806 02:16:49.048568 8933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34487 every 8 connection(s)
I20260806 02:16:49.049530 8861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:49.049731 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 8861
I20260806 02:16:49.049885 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:49.050259 18713 external_mini_cluster.cc:1428] Setting key ef913bb49b7640789afe748785317eef
I20260806 02:16:49.052711 8934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:49.058159 8934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb: Bootstrap starting.
I20260806 02:16:49.059175 8938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:49.051708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:54939 (local address 127.0.0.1:34487)
0806 02:16:49.052149 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:49.052165 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:49.052214 (+ 49us) server_negotiation.cc:407] Connection header received
0806 02:16:49.052927 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:49.052945 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:49.053311 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:49.053645 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:49.054064 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:49.055095 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:49.056033 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:49.056491 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:49.056661 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:49.057560 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:49.057565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:49.057577 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:49.057961 (+ 384us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:49.058012 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:49.058033 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:49.058174 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:49.058348 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 02:16:49.058497 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":150,"threads_started":1}
I20260806 02:16:49.060721 8934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:49.061616 8934 log.cc:824] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:49.062247 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:49.064021 8934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb: No bootstrap required, opened a new log
I20260806 02:16:49.068627 8938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:49.064560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:45635 (local address 127.0.0.1:34487)
0806 02:16:49.064732 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:49.064735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:49.064811 (+ 76us) server_negotiation.cc:407] Connection header received
0806 02:16:49.064943 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:49.064945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:49.064978 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:49.065062 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:49.065357 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:49.065823 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:49.067027 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:49.067226 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:49.067298 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:49.068147 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:49.068150 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:49.068152 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:49.068162 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:49.068184 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:49.068213 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:49.068285 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:49.068443 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 02:16:49.068471 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 02:16:49.068751 8934 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc0f73a598f4846976b2293c8374dfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34487 } }
I20260806 02:16:49.069046 8934 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:49.069118 8934 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdc0f73a598f4846976b2293c8374dfb, State: Initialized, Role: FOLLOWER
I20260806 02:16:49.069581 8934 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc0f73a598f4846976b2293c8374dfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34487 } }
I20260806 02:16:49.069736 8934 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:49.069802 8934 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:49.069907 8934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:49.070791 8934 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc0f73a598f4846976b2293c8374dfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34487 } }
I20260806 02:16:49.071141 8934 leader_election.cc:302] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdc0f73a598f4846976b2293c8374dfb; no voters:
I20260806 02:16:49.071472 8934 leader_election.cc:288] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:49.071602 8945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:49.071848 8945 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [term 1 LEADER]: Becoming Leader. State: Replica: fdc0f73a598f4846976b2293c8374dfb, State: Running, Role: LEADER
I20260806 02:16:49.072222 8945 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc0f73a598f4846976b2293c8374dfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34487 } }
I20260806 02:16:49.072821 8934 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:49.073994 8947 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdc0f73a598f4846976b2293c8374dfb. Latest consensus state: current_term: 1 leader_uuid: "fdc0f73a598f4846976b2293c8374dfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc0f73a598f4846976b2293c8374dfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34487 } } }
I20260806 02:16:49.074132 8947 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:49.074440 8946 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdc0f73a598f4846976b2293c8374dfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc0f73a598f4846976b2293c8374dfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34487 } } }
I20260806 02:16:49.074539 8955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:49.074548 8946 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:49.077545 8955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:49.082850 8955 catalog_manager.cc:1347] Generated new cluster ID: 7be32014d93c44429c3b26371cc20534
I20260806 02:16:49.082931 8955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:49.109737 8955 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:49.110692 8955 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:49.121141 8955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdc0f73a598f4846976b2293c8374dfb: Generated new TSK 0
I20260806 02:16:49.121838 8955 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:49.137192 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 8861
W20260806 02:16:49.145393 8846 connection.cc:535] client connection to 127.0.0.1:34487 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34487): Connection reset by peer (error 104)
2026-08-06T02:16:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:49 dist-test-slave-6k22 krb5kdc[8967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:49 dist-test-slave-6k22 krb5kdc[8967](info): set up 2 sockets
Aug 06 02:16:49 dist-test-slave-6k22 krb5kdc[8967](info): commencing operation
krb5kdc: starting...
W20260806 02:16:50.276993 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:50 dist-test-slave-6k22 krb5kdc[8967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 02:16:50.398149 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40495
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:50.509377 8982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:50.509629 8982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:50.509686 8982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:50.513691 8982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:50.513746 8982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:50.513764 8982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:50.513808 8982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:50.513839 8982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:50.518862 8982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40495
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:50.519943 8982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:50.520936 8982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:50.529096 8988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:50.529098 8990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:50.529103 8987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:50.529433 8982 server_base.cc:1034] running on GCE node
I20260806 02:16:50.529997 8982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:50.530974 8982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:50.532166 8982 hybrid_clock.cc:648] HybridClock initialized: now 1785982610532148 us; error 31 us; skew 500 ppm
Aug 06 02:16:50 dist-test-slave-6k22 krb5kdc[8967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982610, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:50.534874 8982 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 02:16:50.535775 8982 webserver.cc:460] Webserver started at http://127.0.0.1:35603/ using document root <none> and password file <none>
I20260806 02:16:50.536309 8982 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:50.536394 8982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:50.536617 8982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:50.538614 8982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "48c09dd0781c4eb18d94f4f8617c8ae3"
format_stamp: "Formatted at 2026-08-06 02:16:50 on dist-test-slave-6k22"
server_key: "095391daa05e41018e2ac1c18d7e7e71"
server_key_iv: "aefcbeaef67d38c98e2215790793eaaf"
server_key_version: "encryptionkey@0"
I20260806 02:16:50.539176 8982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "48c09dd0781c4eb18d94f4f8617c8ae3"
format_stamp: "Formatted at 2026-08-06 02:16:50 on dist-test-slave-6k22"
server_key: "095391daa05e41018e2ac1c18d7e7e71"
server_key_iv: "aefcbeaef67d38c98e2215790793eaaf"
server_key_version: "encryptionkey@0"
I20260806 02:16:50.542878 8982 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 02:16:50.545296 8997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:50.546442 8982 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 02:16:50.546576 8982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "48c09dd0781c4eb18d94f4f8617c8ae3"
format_stamp: "Formatted at 2026-08-06 02:16:50 on dist-test-slave-6k22"
server_key: "095391daa05e41018e2ac1c18d7e7e71"
server_key_iv: "aefcbeaef67d38c98e2215790793eaaf"
server_key_version: "encryptionkey@0"
I20260806 02:16:50.546681 8982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:50.551301 8982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:50.554054 8982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:50.554250 8982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:50.561015 8982 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40495
I20260806 02:16:50.561074 9051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40495 every 8 connection(s)
I20260806 02:16:50.561949 8982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:50.564404 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 8982
I20260806 02:16:50.564543 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:50.564877 18713 external_mini_cluster.cc:1428] Setting key 2379bbf08a746b2ba400ebeba754545b
I20260806 02:16:50.564889 9053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:50.570231 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:50.566173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:46145 (local address 127.0.0.1:40495)
0806 02:16:50.566577 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:50.566599 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:50.566958 (+ 359us) server_negotiation.cc:407] Connection header received
0806 02:16:50.567674 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:50.567699 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:50.567968 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:50.568260 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:50.569039 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:50.569044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:50.569056 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:50.569065 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:50.569114 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:50.569132 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:50.569243 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:50.569349 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 02:16:50.569522 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":104,"threads_started":1}
I20260806 02:16:50.571864 9053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3: Bootstrap starting.
I20260806 02:16:50.572954 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:50.574477 9053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:50.575397 9053 log.cc:824] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:50.577174 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:50.575256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:48773 (local address 127.0.0.1:40495)
0806 02:16:50.575391 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:50.575396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:50.575637 (+ 241us) server_negotiation.cc:407] Connection header received
0806 02:16:50.575774 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:50.575780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:50.575823 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:50.575914 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:50.576750 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:50.576754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:50.576757 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:50.576769 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:50.576796 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:50.576808 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:50.576899 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:50.577018 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 02:16:50.577058 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 02:16:50.577703 9053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3: No bootstrap required, opened a new log
I20260806 02:16:50.581307 9053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c09dd0781c4eb18d94f4f8617c8ae3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40495 } }
I20260806 02:16:50.581561 9053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:50.581646 9053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c09dd0781c4eb18d94f4f8617c8ae3, State: Initialized, Role: FOLLOWER
I20260806 02:16:50.582093 9053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c09dd0781c4eb18d94f4f8617c8ae3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40495 } }
I20260806 02:16:50.582242 9053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:50.582330 9053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:50.582432 9053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:50.583344 9053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c09dd0781c4eb18d94f4f8617c8ae3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40495 } }
I20260806 02:16:50.583734 9053 leader_election.cc:302] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48c09dd0781c4eb18d94f4f8617c8ae3; no voters:
I20260806 02:16:50.584070 9053 leader_election.cc:288] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:50.584141 9063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:50.584412 9063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [term 1 LEADER]: Becoming Leader. State: Replica: 48c09dd0781c4eb18d94f4f8617c8ae3, State: Running, Role: LEADER
I20260806 02:16:50.584792 9063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c09dd0781c4eb18d94f4f8617c8ae3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40495 } }
I20260806 02:16:50.585343 9053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:50.586458 9065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48c09dd0781c4eb18d94f4f8617c8ae3. Latest consensus state: current_term: 1 leader_uuid: "48c09dd0781c4eb18d94f4f8617c8ae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c09dd0781c4eb18d94f4f8617c8ae3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40495 } } }
I20260806 02:16:50.586634 9065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:50.587000 9064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48c09dd0781c4eb18d94f4f8617c8ae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c09dd0781c4eb18d94f4f8617c8ae3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40495 } } }
I20260806 02:16:50.587069 9072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:50.587184 9064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:50.590338 9072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:50.596336 9072 catalog_manager.cc:1347] Generated new cluster ID: 92f5f2eefdb8434283b01e93cfb75149
I20260806 02:16:50.596408 9072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:50.623688 9072 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:50.624015 9072 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:50.630532 9072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48c09dd0781c4eb18d94f4f8617c8ae3: Generated new TSK 0
I20260806 02:16:50.631175 9072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:50.633911 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 8982
2026-08-06T02:16:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1496 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:50 dist-test-slave-6k22 krb5kdc[9085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:50 dist-test-slave-6k22 krb5kdc[9085](info): set up 2 sockets
Aug 06 02:16:50 dist-test-slave-6k22 krb5kdc[9085](info): commencing operation
krb5kdc: starting...
W20260806 02:16:51.696975 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:51 dist-test-slave-6k22 krb5kdc[9085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:16:51.820132 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:39173
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:33221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:39173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:51.930333 9099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:51.930583 9099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:51.930621 9099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:51.934559 9099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:51.934610 9099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:51.934628 9099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:51.934664 9099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:51.934679 9099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:51.939862 9099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:33221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:39173
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:39173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:51.940814 9099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:51.941774 9099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:51.949806 9107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:51.949817 9104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:51.950073 9099 server_base.cc:1034] running on GCE node
W20260806 02:16:51.949923 9105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:51.950650 9099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:51.951746 9099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:51.952942 9099 hybrid_clock.cc:648] HybridClock initialized: now 1785982611952927 us; error 25 us; skew 500 ppm
Aug 06 02:16:51 dist-test-slave-6k22 krb5kdc[9085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982611, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:51.955742 9099 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:16:51.956600 9099 webserver.cc:460] Webserver started at http://127.18.70.126:34837/ using document root <none> and password file <none>
I20260806 02:16:51.957154 9099 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:51.957242 9099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:51.957470 9099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:51.959426 9099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "fa3d5b66df2640d9b54d6520edbb82e5"
format_stamp: "Formatted at 2026-08-06 02:16:51 on dist-test-slave-6k22"
server_key: "c6eb689f73d696ce77de584ab7dcb01f"
server_key_iv: "5a62ec3afd59ca2fecf15b61bae026b3"
server_key_version: "encryptionkey@0"
I20260806 02:16:51.960036 9099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "fa3d5b66df2640d9b54d6520edbb82e5"
format_stamp: "Formatted at 2026-08-06 02:16:51 on dist-test-slave-6k22"
server_key: "c6eb689f73d696ce77de584ab7dcb01f"
server_key_iv: "5a62ec3afd59ca2fecf15b61bae026b3"
server_key_version: "encryptionkey@0"
I20260806 02:16:51.963515 9099 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:16:51.965955 9114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:51.967058 9099 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:16:51.967200 9099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "fa3d5b66df2640d9b54d6520edbb82e5"
format_stamp: "Formatted at 2026-08-06 02:16:51 on dist-test-slave-6k22"
server_key: "c6eb689f73d696ce77de584ab7dcb01f"
server_key_iv: "5a62ec3afd59ca2fecf15b61bae026b3"
server_key_version: "encryptionkey@0"
I20260806 02:16:51.967307 9099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:51.982671 9099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:51.985417 9099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:51.985611 9099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:51.992760 9099 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:39173
I20260806 02:16:51.992779 9167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:39173 every 8 connection(s)
I20260806 02:16:51.993659 9099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:51.995906 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 9099
I20260806 02:16:51.996048 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:51.996353 18713 external_mini_cluster.cc:1428] Setting key ecc142b559fcbce45df472609df69a35
I20260806 02:16:51.996547 9168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:52.002707 9168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5: Bootstrap starting.
I20260806 02:16:52.005069 9168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5: Neither blocks nor log segments found. Creating new log.
Aug 06 02:16:52 dist-test-slave-6k22 krb5kdc[9085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:16:52.005968 9168 log.cc:824] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:52.008105 9168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5: No bootstrap required, opened a new log
I20260806 02:16:52.010644 9171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:51.997685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33753 (local address 127.18.70.126:39173)
0806 02:16:51.998118 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:51.998132 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:51.998166 (+ 34us) server_negotiation.cc:407] Connection header received
0806 02:16:51.999174 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:51.999205 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:51.999522 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:51.999916 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:52.000725 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:52.001691 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:52.002558 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:52.003089 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:52.003262 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:52.006000 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:52.006005 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:52.006018 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:52.006066 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:52.007778 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:52.008362 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:52.008367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:52.008371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:52.008475 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:52.008809 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:52.008811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:52.008813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:52.009203 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:52.009368 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:52.009696 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 02:16:52.009923 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":153,"threads_started":1}
I20260806 02:16:52.012063 9168 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3d5b66df2640d9b54d6520edbb82e5" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39173 } }
I20260806 02:16:52.012311 9168 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:52.012423 9168 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3d5b66df2640d9b54d6520edbb82e5, State: Initialized, Role: FOLLOWER
I20260806 02:16:52.012893 9168 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3d5b66df2640d9b54d6520edbb82e5" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39173 } }
I20260806 02:16:52.013056 9168 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:52.013145 9168 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:52.013253 9168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:52.013674 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:52.014295 9168 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3d5b66df2640d9b54d6520edbb82e5" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39173 } }
I20260806 02:16:52.014691 9168 leader_election.cc:302] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa3d5b66df2640d9b54d6520edbb82e5; no voters:
I20260806 02:16:52.015020 9168 leader_election.cc:288] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:52.015128 9174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:52.015434 9174 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [term 1 LEADER]: Becoming Leader. State: Replica: fa3d5b66df2640d9b54d6520edbb82e5, State: Running, Role: LEADER
I20260806 02:16:52.015812 9174 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3d5b66df2640d9b54d6520edbb82e5" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39173 } }
I20260806 02:16:52.016362 9168 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:52.016999 9174 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa3d5b66df2640d9b54d6520edbb82e5. Latest consensus state: current_term: 1 leader_uuid: "fa3d5b66df2640d9b54d6520edbb82e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3d5b66df2640d9b54d6520edbb82e5" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39173 } } }
I20260806 02:16:52.017181 9174 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:52.017482 9178 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa3d5b66df2640d9b54d6520edbb82e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3d5b66df2640d9b54d6520edbb82e5" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39173 } } }
I20260806 02:16:52.017628 9178 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:52.017656 9189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:52.020884 9189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:52.026508 9171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:52.015967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46189 (local address 127.18.70.126:39173)
0806 02:16:52.016066 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:52.016070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:52.018891 (+ 2821us) server_negotiation.cc:407] Connection header received
0806 02:16:52.019011 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:52.019016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:52.019061 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:52.019149 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:52.019945 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:52.020480 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:52.021623 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:52.021928 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:52.022043 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:52.023388 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:52.023392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:52.023394 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:52.023417 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:52.024855 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:52.025343 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:52.025348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:52.025349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:52.025425 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:52.025731 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:52.025734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:52.025735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:52.025898 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:52.026038 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:52.026286 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 02:16:52.026369 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 02:16:52.026593 9189 catalog_manager.cc:1347] Generated new cluster ID: 1c6142b4d66a4abca9633dbc12c53c31
I20260806 02:16:52.026678 9189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:52.033895 9189 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:52.034629 9189 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:52.045394 9189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa3d5b66df2640d9b54d6520edbb82e5: Generated new TSK 0
I20260806 02:16:52.046011 9189 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:52.053773 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 9099
W20260806 02:16:52.062057 9083 connection.cc:535] client connection to 127.18.70.126:39173 recv error: Network error: failed to read from TLS socket (remote: 127.18.70.126:39173): Connection reset by peer (error 104)
2026-08-06T02:16:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:52 dist-test-slave-6k22 krb5kdc[9203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:52 dist-test-slave-6k22 krb5kdc[9203](info): set up 2 sockets
Aug 06 02:16:52 dist-test-slave-6k22 krb5kdc[9203](info): commencing operation
krb5kdc: starting...
W20260806 02:16:53.101123 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:53 dist-test-slave-6k22 krb5kdc[9203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:16:53.222585 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:40799
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:40799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:53.329761 9216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:53.330077 9216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:53.330153 9216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:53.334080 9216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:53.334172 9216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:53.334206 9216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:53.334244 9216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:53.334278 9216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:53.339335 9216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:40799
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:40799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:53.340343 9216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:53.341318 9216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:53.349537 9225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:53.349534 9221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:53.349537 9222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:53.349746 9216 server_base.cc:1034] running on GCE node
I20260806 02:16:53.350425 9216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:53.351487 9216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:53.352650 9216 hybrid_clock.cc:648] HybridClock initialized: now 1785982613352635 us; error 30 us; skew 500 ppm
Aug 06 02:16:53 dist-test-slave-6k22 krb5kdc[9203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982613, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:53.355151 9216 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:16:53.356027 9216 webserver.cc:460] Webserver started at http://127.18.70.126:39837/ using document root <none> and password file <none>
I20260806 02:16:53.356560 9216 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:53.356645 9216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:53.356894 9216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:53.358983 9216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "bf3bf741950e4ef58a99e7a1f6d9fa1a"
format_stamp: "Formatted at 2026-08-06 02:16:53 on dist-test-slave-6k22"
server_key: "9aaf6fe25880039b8c1bb23c128227a0"
server_key_iv: "193c29cbf426377242cc700509299417"
server_key_version: "encryptionkey@0"
I20260806 02:16:53.359537 9216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "bf3bf741950e4ef58a99e7a1f6d9fa1a"
format_stamp: "Formatted at 2026-08-06 02:16:53 on dist-test-slave-6k22"
server_key: "9aaf6fe25880039b8c1bb23c128227a0"
server_key_iv: "193c29cbf426377242cc700509299417"
server_key_version: "encryptionkey@0"
I20260806 02:16:53.363093 9216 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:16:53.365403 9232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:53.366518 9216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 02:16:53.366659 9216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "bf3bf741950e4ef58a99e7a1f6d9fa1a"
format_stamp: "Formatted at 2026-08-06 02:16:53 on dist-test-slave-6k22"
server_key: "9aaf6fe25880039b8c1bb23c128227a0"
server_key_iv: "193c29cbf426377242cc700509299417"
server_key_version: "encryptionkey@0"
I20260806 02:16:53.366761 9216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:53.375785 9216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:53.378547 9216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:53.378739 9216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:53.385532 9216 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:40799
I20260806 02:16:53.385618 9287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:40799 every 8 connection(s)
I20260806 02:16:53.386538 9216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:53.388108 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 9216
I20260806 02:16:53.388252 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:53.388576 18713 external_mini_cluster.cc:1428] Setting key b08545c872aa29b1a631981638a80d8a
I20260806 02:16:53.389879 9288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:53.396788 9288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a: Bootstrap starting.
I20260806 02:16:53.398809 9291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:53.389904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50065 (local address 127.18.70.126:40799)
0806 02:16:53.390370 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:53.390390 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:53.390433 (+ 43us) server_negotiation.cc:407] Connection header received
0806 02:16:53.391266 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:53.391288 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:53.391752 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:53.392113 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:53.392509 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:53.394029 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:53.394962 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:53.395686 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:53.395864 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:53.396770 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:53.396776 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:53.396791 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:53.397298 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:53.397364 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:53.397394 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:53.397525 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:53.397708 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 02:16:53.397847 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":121,"threads_started":1}
I20260806 02:16:53.399629 9288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:53.400400 9288 log.cc:824] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:53.401880 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:53.402563 9288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a: No bootstrap required, opened a new log
I20260806 02:16:53.407155 9288 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf3bf741950e4ef58a99e7a1f6d9fa1a" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40799 } }
I20260806 02:16:53.407464 9288 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:53.407568 9288 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf3bf741950e4ef58a99e7a1f6d9fa1a, State: Initialized, Role: FOLLOWER
I20260806 02:16:53.408056 9291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:53.403869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38025 (local address 127.18.70.126:40799)
0806 02:16:53.403975 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:53.403979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:53.404174 (+ 195us) server_negotiation.cc:407] Connection header received
0806 02:16:53.404278 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:53.404292 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:53.404325 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:53.404389 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:53.404674 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:53.405161 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:53.406370 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:53.406599 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:53.406696 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:53.407587 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:53.407594 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:53.407600 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:53.407614 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:53.407641 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:53.407653 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:53.407748 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:53.407912 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 02:16:53.407953 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 02:16:53.408102 9288 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf3bf741950e4ef58a99e7a1f6d9fa1a" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40799 } }
I20260806 02:16:53.408278 9288 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:53.408352 9288 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:53.408455 9288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:53.409520 9288 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf3bf741950e4ef58a99e7a1f6d9fa1a" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40799 } }
I20260806 02:16:53.409813 9288 leader_election.cc:302] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf3bf741950e4ef58a99e7a1f6d9fa1a; no voters:
I20260806 02:16:53.410065 9288 leader_election.cc:288] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:53.410181 9299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:53.410403 9299 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [term 1 LEADER]: Becoming Leader. State: Replica: bf3bf741950e4ef58a99e7a1f6d9fa1a, State: Running, Role: LEADER
I20260806 02:16:53.410737 9299 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf3bf741950e4ef58a99e7a1f6d9fa1a" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40799 } }
I20260806 02:16:53.411314 9288 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:53.412468 9301 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf3bf741950e4ef58a99e7a1f6d9fa1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf3bf741950e4ef58a99e7a1f6d9fa1a" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40799 } } }
I20260806 02:16:53.412606 9301 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:53.413219 9302 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf3bf741950e4ef58a99e7a1f6d9fa1a. Latest consensus state: current_term: 1 leader_uuid: "bf3bf741950e4ef58a99e7a1f6d9fa1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf3bf741950e4ef58a99e7a1f6d9fa1a" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40799 } } }
I20260806 02:16:53.413379 9302 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a [sys.catalog]: This master's current role is: LEADER
W20260806 02:16:53.416036 9316 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:16:53.416124 9316 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:16:53.416260 9309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:53.417130 9309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:53.422358 9309 catalog_manager.cc:1347] Generated new cluster ID: 5913fbcf0c914266bc32a4e5114791cb
I20260806 02:16:53.422420 9309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:53.441676 9309 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:53.442886 9309 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:53.450968 9309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf3bf741950e4ef58a99e7a1f6d9fa1a: Generated new TSK 0
I20260806 02:16:53.451752 9309 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:53.461079 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 9216
W20260806 02:16:53.469017 9201 connection.cc:535] client connection to 127.18.70.126:40799 recv error: Network error: failed to read from TLS socket (remote: 127.18.70.126:40799): Connection reset by peer (error 104)
2026-08-06T02:16:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1406 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:53 dist-test-slave-6k22 krb5kdc[9322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:53 dist-test-slave-6k22 krb5kdc[9322](info): set up 2 sockets
Aug 06 02:16:53 dist-test-slave-6k22 krb5kdc[9322](info): commencing operation
krb5kdc: starting...
W20260806 02:16:54.529773 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:54 dist-test-slave-6k22 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:16:54.654210 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:40169
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:40169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:54.765949 9336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:54.766206 9336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:54.766281 9336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:54.770040 9336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:54.770113 9336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:54.770148 9336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:54.770203 9336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:54.770238 9336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:54.775389 9336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:40169
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:40169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:54.776366 9336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:54.777302 9336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:54.785764 9342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:54.785816 9343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:54.785765 9345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:54.785806 9336 server_base.cc:1034] running on GCE node
I20260806 02:16:54.786513 9336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:54.787523 9336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:54.788734 9336 hybrid_clock.cc:648] HybridClock initialized: now 1785982614788720 us; error 25 us; skew 500 ppm
Aug 06 02:16:54 dist-test-slave-6k22 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982614, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:54.796011 9336 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:16:54.796852 9336 webserver.cc:460] Webserver started at http://127.18.70.126:41561/ using document root <none> and password file <none>
I20260806 02:16:54.797438 9336 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:54.797530 9336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:54.797770 9336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:54.799880 9336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "8fd6be4f768846c1861ddca5d9e45062"
format_stamp: "Formatted at 2026-08-06 02:16:54 on dist-test-slave-6k22"
server_key: "bf054607eea773f3461caddd81160768"
server_key_iv: "56ebdf7f026a8038aa08c19efda89335"
server_key_version: "encryptionkey@0"
I20260806 02:16:54.800459 9336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "8fd6be4f768846c1861ddca5d9e45062"
format_stamp: "Formatted at 2026-08-06 02:16:54 on dist-test-slave-6k22"
server_key: "bf054607eea773f3461caddd81160768"
server_key_iv: "56ebdf7f026a8038aa08c19efda89335"
server_key_version: "encryptionkey@0"
I20260806 02:16:54.804011 9336 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 02:16:54.806424 9353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:54.807638 9336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 02:16:54.807771 9336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "8fd6be4f768846c1861ddca5d9e45062"
format_stamp: "Formatted at 2026-08-06 02:16:54 on dist-test-slave-6k22"
server_key: "bf054607eea773f3461caddd81160768"
server_key_iv: "56ebdf7f026a8038aa08c19efda89335"
server_key_version: "encryptionkey@0"
I20260806 02:16:54.807876 9336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:54.812270 9336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:54.815419 9336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:54.815687 9336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:54.822760 9336 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:40169
I20260806 02:16:54.822824 9417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:40169 every 8 connection(s)
I20260806 02:16:54.823787 9336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:54.826577 9418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:54.830672 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 9336
I20260806 02:16:54.830832 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:54.831105 18713 external_mini_cluster.cc:1428] Setting key 952f6c2dc48d59d96c3687f7ab3c2d42
I20260806 02:16:54.833640 9418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062: Bootstrap starting.
I20260806 02:16:54.836387 9421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:54.832593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53893 (local address 127.18.70.126:40169)
0806 02:16:54.833079 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:54.833095 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:54.833129 (+ 34us) server_negotiation.cc:407] Connection header received
0806 02:16:54.833793 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:54.833813 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:54.834085 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:54.834406 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:54.835187 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:54.835193 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:54.835224 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:54.835234 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:54.835288 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:54.835307 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:54.835475 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:54.835545 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 02:16:54.835733 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":153,"threads_started":1}
I20260806 02:16:54.836431 9418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:54.837296 9418 log.cc:824] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:54.839460 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:54.839835 9418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062: No bootstrap required, opened a new log
I20260806 02:16:54.843245 9421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:54.841596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46255 (local address 127.18.70.126:40169)
0806 02:16:54.841716 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:54.841719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:54.841901 (+ 182us) server_negotiation.cc:407] Connection header received
0806 02:16:54.842050 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:54.842051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:54.842077 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:54.842132 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:54.842874 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:54.842877 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:54.842879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:54.842889 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:54.842911 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:54.842920 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:54.842986 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:54.843109 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 02:16:54.843137 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 02:16:54.844843 9418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd6be4f768846c1861ddca5d9e45062" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40169 } }
I20260806 02:16:54.845108 9418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:54.845180 9418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd6be4f768846c1861ddca5d9e45062, State: Initialized, Role: FOLLOWER
I20260806 02:16:54.845671 9418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd6be4f768846c1861ddca5d9e45062" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40169 } }
I20260806 02:16:54.845849 9418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:54.845955 9418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:54.846083 9418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:54.847049 9418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd6be4f768846c1861ddca5d9e45062" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40169 } }
I20260806 02:16:54.847446 9418 leader_election.cc:302] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fd6be4f768846c1861ddca5d9e45062; no voters:
I20260806 02:16:54.847911 9418 leader_election.cc:288] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:54.847992 9428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:54.848251 9428 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd6be4f768846c1861ddca5d9e45062, State: Running, Role: LEADER
I20260806 02:16:54.848567 9428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd6be4f768846c1861ddca5d9e45062" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40169 } }
I20260806 02:16:54.849320 9418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:54.849903 9428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fd6be4f768846c1861ddca5d9e45062. Latest consensus state: current_term: 1 leader_uuid: "8fd6be4f768846c1861ddca5d9e45062" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd6be4f768846c1861ddca5d9e45062" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40169 } } }
I20260806 02:16:54.850138 9428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:54.850299 9430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fd6be4f768846c1861ddca5d9e45062" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd6be4f768846c1861ddca5d9e45062" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 40169 } } }
I20260806 02:16:54.850457 9430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:54.850749 9436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:54.853762 9436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:54.859874 9436 catalog_manager.cc:1347] Generated new cluster ID: d87943fd958241fd96f80df7bfd9756c
I20260806 02:16:54.859930 9436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:54.877727 9436 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:54.878018 9436 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:54.884855 9436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fd6be4f768846c1861ddca5d9e45062: Generated new TSK 0
I20260806 02:16:54.885530 9436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:54.888867 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 9336
2026-08-06T02:16:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:54 dist-test-slave-6k22 krb5kdc[9450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:54 dist-test-slave-6k22 krb5kdc[9450](info): set up 2 sockets
Aug 06 02:16:54 dist-test-slave-6k22 krb5kdc[9450](info): commencing operation
krb5kdc: starting...
W20260806 02:16:55.936015 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:56 dist-test-slave-6k22 krb5kdc[9450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:16:56.073935 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:39305
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:34553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:39305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:56.187644 9465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:56.188010 9465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:56.188097 9465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:56.192374 9465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:56.192430 9465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:56.192448 9465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:56.192495 9465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:56.192526 9465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:56.197651 9465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:34553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:39305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:39305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:56.198671 9465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:56.199746 9465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:56.207772 9471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:56.207784 9470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:56.207782 9473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:56.208117 9465 server_base.cc:1034] running on GCE node
I20260806 02:16:56.208596 9465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:56.209563 9465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:56.210772 9465 hybrid_clock.cc:648] HybridClock initialized: now 1785982616210752 us; error 30 us; skew 500 ppm
Aug 06 02:16:56 dist-test-slave-6k22 krb5kdc[9450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982616, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:56.213387 9465 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:16:56.214200 9465 webserver.cc:460] Webserver started at http://127.18.70.126:38021/ using document root <none> and password file <none>
I20260806 02:16:56.214743 9465 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:56.214826 9465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:56.215039 9465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:56.216955 9465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "982c6f1e70124dec99ee3b08cac7ffd1"
format_stamp: "Formatted at 2026-08-06 02:16:56 on dist-test-slave-6k22"
server_key: "a8b8131af16850a083d241d2f102e0fc"
server_key_iv: "7d2b4d299c3865f314741b9a853170e1"
server_key_version: "encryptionkey@0"
I20260806 02:16:56.217494 9465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "982c6f1e70124dec99ee3b08cac7ffd1"
format_stamp: "Formatted at 2026-08-06 02:16:56 on dist-test-slave-6k22"
server_key: "a8b8131af16850a083d241d2f102e0fc"
server_key_iv: "7d2b4d299c3865f314741b9a853170e1"
server_key_version: "encryptionkey@0"
I20260806 02:16:56.221084 9465 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 02:16:56.223287 9480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:56.224529 9465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:16:56.224658 9465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "982c6f1e70124dec99ee3b08cac7ffd1"
format_stamp: "Formatted at 2026-08-06 02:16:56 on dist-test-slave-6k22"
server_key: "a8b8131af16850a083d241d2f102e0fc"
server_key_iv: "7d2b4d299c3865f314741b9a853170e1"
server_key_version: "encryptionkey@0"
I20260806 02:16:56.224769 9465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:56.236690 9465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:56.239208 9465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:56.239385 9465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:56.246245 9465 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:39305
I20260806 02:16:56.246272 9538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:39305 every 8 connection(s)
I20260806 02:16:56.247144 9465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:56.249786 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 9465
I20260806 02:16:56.249922 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:56.249967 9539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:56.250218 18713 external_mini_cluster.cc:1428] Setting key 82923930db427a8aa9f86bf8db28cad6
I20260806 02:16:56.256724 9539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1: Bootstrap starting.
I20260806 02:16:56.258889 9539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:56.259747 9539 log.cc:824] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1: Log is configured to *not* fsync() on all Append() calls
Aug 06 02:16:56 dist-test-slave-6k22 krb5kdc[9450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:16:56.261585 9539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1: No bootstrap required, opened a new log
I20260806 02:16:56.264997 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:56.251679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58675 (local address 127.18.70.126:39305)
0806 02:16:56.252133 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:56.252150 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:56.252191 (+ 41us) server_negotiation.cc:407] Connection header received
0806 02:16:56.252827 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:56.252850 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:56.253260 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:56.253610 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:56.254512 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:56.255693 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:56.256701 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:56.257314 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:56.257517 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:56.260536 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:56.260541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:56.260553 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:56.260579 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:56.262250 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:56.262703 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:56.262707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:56.262711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:56.262811 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:56.263091 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:56.263093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:56.263094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:56.263522 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:56.263752 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:56.264044 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 02:16:56.264308 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":126,"threads_started":1}
I20260806 02:16:56.265511 9539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c6f1e70124dec99ee3b08cac7ffd1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39305 } }
I20260806 02:16:56.265734 9539 raft_consensus.cc:374] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:56.265807 9539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982c6f1e70124dec99ee3b08cac7ffd1, State: Initialized, Role: FOLLOWER
I20260806 02:16:56.266248 9539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c6f1e70124dec99ee3b08cac7ffd1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39305 } }
I20260806 02:16:56.266390 9539 raft_consensus.cc:388] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:56.266467 9539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:56.266563 9539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:56.267586 9539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c6f1e70124dec99ee3b08cac7ffd1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39305 } }
I20260806 02:16:56.267720 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:56.267951 9539 leader_election.cc:302] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 982c6f1e70124dec99ee3b08cac7ffd1; no voters:
I20260806 02:16:56.268287 9539 leader_election.cc:288] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:56.268399 9544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:56.268637 9544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [term 1 LEADER]: Becoming Leader. State: Replica: 982c6f1e70124dec99ee3b08cac7ffd1, State: Running, Role: LEADER
I20260806 02:16:56.268930 9544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c6f1e70124dec99ee3b08cac7ffd1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39305 } }
I20260806 02:16:56.269618 9539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:56.270484 9545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "982c6f1e70124dec99ee3b08cac7ffd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c6f1e70124dec99ee3b08cac7ffd1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39305 } } }
I20260806 02:16:56.270596 9545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:56.270958 9546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 982c6f1e70124dec99ee3b08cac7ffd1. Latest consensus state: current_term: 1 leader_uuid: "982c6f1e70124dec99ee3b08cac7ffd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c6f1e70124dec99ee3b08cac7ffd1" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39305 } } }
I20260806 02:16:56.271011 9557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:56.271062 9546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:56.274300 9557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:56.279881 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:56.269934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56971 (local address 127.18.70.126:39305)
0806 02:16:56.270039 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:56.270043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:56.271511 (+ 1468us) server_negotiation.cc:407] Connection header received
0806 02:16:56.271714 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:56.271717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:56.271760 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:56.271809 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:56.272728 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:56.273515 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:56.274624 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:56.274952 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:56.275052 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:56.276750 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:56.276756 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:56.276759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:16:56.276789 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:56.278246 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:56.278671 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:56.278674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:56.278676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:56.278754 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:16:56.279024 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:16:56.279028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:16:56.279030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:16:56.279201 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:56.279336 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:56.279585 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 02:16:56.279709 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 02:16:56.281431 9557 catalog_manager.cc:1347] Generated new cluster ID: 5301a2176ae24e67b78509797d1b8133
I20260806 02:16:56.281497 9557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:56.290555 9557 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:56.291874 9557 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:56.302500 9557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 982c6f1e70124dec99ee3b08cac7ffd1: Generated new TSK 0
I20260806 02:16:56.303220 9557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:56.311062 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 9465
2026-08-06T02:16:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:56 dist-test-slave-6k22 krb5kdc[9572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:56 dist-test-slave-6k22 krb5kdc[9572](info): set up 2 sockets
Aug 06 02:16:56 dist-test-slave-6k22 krb5kdc[9572](info): commencing operation
krb5kdc: starting...
W20260806 02:16:57.365159 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:57 dist-test-slave-6k22 krb5kdc[9572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:16:57.486620 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:39689
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:45223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:39689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:57.596145 9585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:57.596369 9585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:57.596406 9585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:57.600229 9585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:57.600275 9585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:57.600294 9585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:57.600310 9585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:57.600327 9585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:57.605290 9585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:45223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:39689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:39689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:57.606206 9585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:57.607107 9585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:57.615448 9590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:57.615468 9594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:57.615576 9585 server_base.cc:1034] running on GCE node
W20260806 02:16:57.615468 9591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:57.616293 9585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:57.617360 9585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:57.618526 9585 hybrid_clock.cc:648] HybridClock initialized: now 1785982617618492 us; error 46 us; skew 500 ppm
Aug 06 02:16:57 dist-test-slave-6k22 krb5kdc[9572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982617, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:57.621208 9585 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:16:57.622009 9585 webserver.cc:460] Webserver started at http://127.18.70.126:36647/ using document root <none> and password file <none>
I20260806 02:16:57.622550 9585 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:57.622605 9585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:57.622766 9585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:57.624861 9585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "4c46924694bf4b6abe32770ee073f601"
format_stamp: "Formatted at 2026-08-06 02:16:57 on dist-test-slave-6k22"
server_key: "e5f12ae3ec5d9f168e3d98e94564ddfa"
server_key_iv: "1aa1a07e9911ccdf3256c2084767a054"
server_key_version: "encryptionkey@0"
I20260806 02:16:57.625381 9585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "4c46924694bf4b6abe32770ee073f601"
format_stamp: "Formatted at 2026-08-06 02:16:57 on dist-test-slave-6k22"
server_key: "e5f12ae3ec5d9f168e3d98e94564ddfa"
server_key_iv: "1aa1a07e9911ccdf3256c2084767a054"
server_key_version: "encryptionkey@0"
I20260806 02:16:57.628931 9585 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 02:16:57.631167 9603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:57.632404 9585 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:16:57.632529 9585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "4c46924694bf4b6abe32770ee073f601"
format_stamp: "Formatted at 2026-08-06 02:16:57 on dist-test-slave-6k22"
server_key: "e5f12ae3ec5d9f168e3d98e94564ddfa"
server_key_iv: "1aa1a07e9911ccdf3256c2084767a054"
server_key_version: "encryptionkey@0"
I20260806 02:16:57.632622 9585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:57.646060 9585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:57.648849 9585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:57.649055 9585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:57.656104 9585 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:39689
I20260806 02:16:57.656134 9659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:39689 every 8 connection(s)
I20260806 02:16:57.657160 9585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:57.659830 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:57.662478 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 9585
I20260806 02:16:57.662627 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:57.662963 18713 external_mini_cluster.cc:1428] Setting key cfdb00c9c677b53ca417b2c36f4ef7d0
I20260806 02:16:57.665845 9660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601: Bootstrap starting.
I20260806 02:16:57.668102 9660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:57.668787 9660 log.cc:824] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:57.670578 9660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601: No bootstrap required, opened a new log
I20260806 02:16:57.671542 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:57.664313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59943 (local address 127.18.70.126:39689)
0806 02:16:57.664763 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:57.664777 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:57.664812 (+ 35us) server_negotiation.cc:407] Connection header received
0806 02:16:57.665412 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:57.665429 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:57.665746 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:57.666069 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:57.666482 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:57.667446 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:57.668415 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:57.668861 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:57.669083 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:57.669881 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:57.669886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:57.669898 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:57.670257 (+ 359us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:57.670308 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:57.670329 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:57.670424 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:57.670574 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 02:16:57.670778 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":122,"threads_started":1}
I20260806 02:16:57.674382 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:57.674683 9660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c46924694bf4b6abe32770ee073f601" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39689 } }
I20260806 02:16:57.674961 9660 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:57.675021 9660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c46924694bf4b6abe32770ee073f601, State: Initialized, Role: FOLLOWER
I20260806 02:16:57.675489 9660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c46924694bf4b6abe32770ee073f601" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39689 } }
I20260806 02:16:57.675671 9660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:57.675761 9660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:57.675892 9660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:57.677052 9660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c46924694bf4b6abe32770ee073f601" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39689 } }
I20260806 02:16:57.677429 9660 leader_election.cc:302] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c46924694bf4b6abe32770ee073f601; no voters:
I20260806 02:16:57.677781 9660 leader_election.cc:288] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:57.677901 9671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:57.678078 9671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [term 1 LEADER]: Becoming Leader. State: Replica: 4c46924694bf4b6abe32770ee073f601, State: Running, Role: LEADER
I20260806 02:16:57.678467 9671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c46924694bf4b6abe32770ee073f601" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39689 } }
I20260806 02:16:57.679040 9660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:57.679643 9671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c46924694bf4b6abe32770ee073f601. Latest consensus state: current_term: 1 leader_uuid: "4c46924694bf4b6abe32770ee073f601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c46924694bf4b6abe32770ee073f601" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39689 } } }
I20260806 02:16:57.680433 9671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:57.680300 9672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c46924694bf4b6abe32770ee073f601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c46924694bf4b6abe32770ee073f601" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39689 } } }
I20260806 02:16:57.680742 9672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:57.681229 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:57.676738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54655 (local address 127.18.70.126:39689)
0806 02:16:57.676838 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:57.676842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:57.677077 (+ 235us) server_negotiation.cc:407] Connection header received
0806 02:16:57.677216 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:57.677219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:57.677264 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:57.677329 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:57.677674 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:57.678305 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:57.679211 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:16:57.679426 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:16:57.679536 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:16:57.680551 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:57.680557 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:57.680563 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:57.680578 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:57.680604 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:57.680619 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:57.680686 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:57.680997 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 02:16:57.681067 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 02:16:57.684508 9688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:16:57.684568 9688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:16:57.684624 9682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:57.685355 9682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:57.690407 9682 catalog_manager.cc:1347] Generated new cluster ID: 4bc6b163d5cb49f8854be1f0741e276c
I20260806 02:16:57.690469 9682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:57.709233 9682 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:57.710429 9682 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:57.731791 9682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c46924694bf4b6abe32770ee073f601: Generated new TSK 0
I20260806 02:16:57.732524 9682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:57.741189 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 9585
2026-08-06T02:16:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:57 dist-test-slave-6k22 krb5kdc[9695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:57 dist-test-slave-6k22 krb5kdc[9695](info): set up 2 sockets
Aug 06 02:16:57 dist-test-slave-6k22 krb5kdc[9695](info): commencing operation
krb5kdc: starting...
W20260806 02:16:58.800711 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:16:58 dist-test-slave-6k22 krb5kdc[9695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:16:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:16:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:16:58.926198 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:43431
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:41531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:43431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:16:59.039469 9709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:16:59.039830 9709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:16:59.039873 9709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:16:59.043726 9709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:16:59.043780 9709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:16:59.043802 9709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:16:59.043821 9709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:16:59.043839 9709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:16:59.048857 9709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:41531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:43431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:43431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:16:59.049809 9709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:16:59.050797 9709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:16:59.059458 9715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:16:59.059448 9714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:59.059508 9709 server_base.cc:1034] running on GCE node
W20260806 02:16:59.059446 9717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:16:59.060333 9709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:16:59.061398 9709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:16:59.062601 9709 hybrid_clock.cc:648] HybridClock initialized: now 1785982619062583 us; error 29 us; skew 500 ppm
Aug 06 02:16:59 dist-test-slave-6k22 krb5kdc[9695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982619, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:16:59.065191 9709 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:16:59.066013 9709 webserver.cc:460] Webserver started at http://127.18.70.126:37571/ using document root <none> and password file <none>
I20260806 02:16:59.066546 9709 fs_manager.cc:359] Metadata directory not provided
I20260806 02:16:59.066630 9709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:16:59.066847 9709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:16:59.068874 9709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "0594396630ab46dd900094c0e23e2c31"
format_stamp: "Formatted at 2026-08-06 02:16:59 on dist-test-slave-6k22"
server_key: "42c1e6bab20eb39d9ec1f69d3663d3d9"
server_key_iv: "b3df2a6a9d76fabd89e8bd29a7ecac0e"
server_key_version: "encryptionkey@0"
I20260806 02:16:59.069432 9709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "0594396630ab46dd900094c0e23e2c31"
format_stamp: "Formatted at 2026-08-06 02:16:59 on dist-test-slave-6k22"
server_key: "42c1e6bab20eb39d9ec1f69d3663d3d9"
server_key_iv: "b3df2a6a9d76fabd89e8bd29a7ecac0e"
server_key_version: "encryptionkey@0"
I20260806 02:16:59.073154 9709 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 02:16:59.075428 9725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:16:59.076713 9709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:16:59.076850 9709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "0594396630ab46dd900094c0e23e2c31"
format_stamp: "Formatted at 2026-08-06 02:16:59 on dist-test-slave-6k22"
server_key: "42c1e6bab20eb39d9ec1f69d3663d3d9"
server_key_iv: "b3df2a6a9d76fabd89e8bd29a7ecac0e"
server_key_version: "encryptionkey@0"
I20260806 02:16:59.076953 9709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:16:59.081338 9709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:16:59.084034 9709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:16:59.084216 9709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:16:59.090636 9709 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:43431
I20260806 02:16:59.090668 9784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:43431 every 8 connection(s)
I20260806 02:16:59.091531 9709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:16:59.092144 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 9709
I20260806 02:16:59.092295 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:16:59.092584 18713 external_mini_cluster.cc:1428] Setting key 68ebcc90982499b7b4ebdcb71c49f9f3
I20260806 02:16:59.094704 9785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:16:59.097837 9788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:59.093855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54021 (local address 127.18.70.126:43431)
0806 02:16:59.094264 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:59.094283 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:59.094387 (+ 104us) server_negotiation.cc:407] Connection header received
0806 02:16:59.095242 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:59.095265 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:59.095618 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:59.095962 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:59.096572 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:59.096578 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:59.096591 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:59.096600 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:59.096651 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:59.096675 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:59.096808 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:59.096871 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 02:16:59.097008 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":112,"threads_started":1}
I20260806 02:16:59.100603 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:16:59.101382 9785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31: Bootstrap starting.
I20260806 02:16:59.104101 9785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31: Neither blocks nor log segments found. Creating new log.
I20260806 02:16:59.104373 9788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:16:59.102853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40435 (local address 127.18.70.126:43431)
0806 02:16:59.102964 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 02:16:59.102968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:16:59.103131 (+ 163us) server_negotiation.cc:407] Connection header received
0806 02:16:59.103226 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:16:59.103229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:16:59.103258 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:16:59.103323 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:16:59.103955 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:16:59.103961 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:16:59.103967 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 02:16:59.103982 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:16:59.104011 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 02:16:59.104025 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:16:59.104132 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 02:16:59.104210 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 02:16:59.104247 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 02:16:59.104928 9785 log.cc:824] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31: Log is configured to *not* fsync() on all Append() calls
I20260806 02:16:59.106684 9785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31: No bootstrap required, opened a new log
I20260806 02:16:59.109915 9785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0594396630ab46dd900094c0e23e2c31" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 43431 } }
I20260806 02:16:59.110111 9785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 02:16:59.110153 9785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0594396630ab46dd900094c0e23e2c31, State: Initialized, Role: FOLLOWER
I20260806 02:16:59.110488 9785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0594396630ab46dd900094c0e23e2c31" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 43431 } }
I20260806 02:16:59.110617 9785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 02:16:59.110685 9785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 02:16:59.110821 9785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:16:59.111712 9785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0594396630ab46dd900094c0e23e2c31" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 43431 } }
I20260806 02:16:59.112023 9785 leader_election.cc:302] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0594396630ab46dd900094c0e23e2c31; no voters:
I20260806 02:16:59.112313 9785 leader_election.cc:288] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 02:16:59.112422 9796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:16:59.112649 9796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [term 1 LEADER]: Becoming Leader. State: Replica: 0594396630ab46dd900094c0e23e2c31, State: Running, Role: LEADER
I20260806 02:16:59.112965 9796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0594396630ab46dd900094c0e23e2c31" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 43431 } }
I20260806 02:16:59.113406 9785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:16:59.114547 9798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0594396630ab46dd900094c0e23e2c31. Latest consensus state: current_term: 1 leader_uuid: "0594396630ab46dd900094c0e23e2c31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0594396630ab46dd900094c0e23e2c31" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 43431 } } }
I20260806 02:16:59.114662 9798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [sys.catalog]: This master's current role is: LEADER
I20260806 02:16:59.115048 9797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0594396630ab46dd900094c0e23e2c31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0594396630ab46dd900094c0e23e2c31" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 43431 } } }
I20260806 02:16:59.115185 9797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31 [sys.catalog]: This master's current role is: LEADER
W20260806 02:16:59.117972 9812 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:16:59.118052 9812 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:16:59.118307 9805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:16:59.119246 9805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:16:59.124331 9805 catalog_manager.cc:1347] Generated new cluster ID: f85800f8f50c43e5a347fe6e52d3e042
I20260806 02:16:59.124388 9805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:16:59.160000 9805 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:16:59.160331 9805 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:16:59.170413 9805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0594396630ab46dd900094c0e23e2c31: Generated new TSK 0
I20260806 02:16:59.171173 9805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 02:16:59.179039 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 9709
2026-08-06T02:16:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1437 ms)
[----------] 18 tests from AuthTokenIssuingTest (25903 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:16:59 dist-test-slave-6k22 krb5kdc[9818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:16:59 dist-test-slave-6k22 krb5kdc[9818](info): set up 2 sockets
Aug 06 02:16:59 dist-test-slave-6k22 krb5kdc[9818](info): commencing operation
krb5kdc: starting...
W20260806 02:17:00.237142 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:17:00 dist-test-slave-6k22 krb5kdc[9818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:17:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:17:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:17:00.359992 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:32915
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:42147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:32915,127.18.70.125:37237,127.18.70.124:40673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:17:00.468936 9834 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 02:17:00.469216 9834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:17:00.469238 9834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:17:00.469275 9834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:17:00.473126 9834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:17:00.473176 9834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:17:00.473196 9834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:17:00.473212 9834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:17:00.473227 9834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:17:00.478320 9834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:42147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:32915,127.18.70.125:37237,127.18.70.124:40673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:32915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:17:00.479249 9834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:17:00.480293 9834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:17:00.488485 9840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:17:00.488453 9839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:17:00.488453 9842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:17:00.488822 9834 server_base.cc:1034] running on GCE node
I20260806 02:17:00.489344 9834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:17:00.490418 9834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:17:00.491631 9834 hybrid_clock.cc:648] HybridClock initialized: now 1785982620491613 us; error 24 us; skew 500 ppm
Aug 06 02:17:00 dist-test-slave-6k22 krb5kdc[9818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:17:00.499388 9834 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:17:00.500380 9834 webserver.cc:460] Webserver started at http://127.18.70.126:34639/ using document root <none> and password file <none>
I20260806 02:17:00.500960 9834 fs_manager.cc:359] Metadata directory not provided
I20260806 02:17:00.501050 9834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:17:00.501283 9834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:17:00.503279 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "f5a5297f61f64240a87031da280c33f2"
format_stamp: "Formatted at 2026-08-06 02:17:00 on dist-test-slave-6k22"
server_key: "28a792744fb11d799579849fe0837f33"
server_key_iv: "30927eabb8de10b6534ee9ab53f50b83"
server_key_version: "encryptionkey@0"
I20260806 02:17:00.503844 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "f5a5297f61f64240a87031da280c33f2"
format_stamp: "Formatted at 2026-08-06 02:17:00 on dist-test-slave-6k22"
server_key: "28a792744fb11d799579849fe0837f33"
server_key_iv: "30927eabb8de10b6534ee9ab53f50b83"
server_key_version: "encryptionkey@0"
I20260806 02:17:00.507400 9834 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 02:17:00.509948 9849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:17:00.511211 9834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:17:00.511344 9834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "f5a5297f61f64240a87031da280c33f2"
format_stamp: "Formatted at 2026-08-06 02:17:00 on dist-test-slave-6k22"
server_key: "28a792744fb11d799579849fe0837f33"
server_key_iv: "30927eabb8de10b6534ee9ab53f50b83"
server_key_version: "encryptionkey@0"
I20260806 02:17:00.511446 9834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:17:00.527387 9834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:17:00.530530 9834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:17:00.530727 9834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:17:00.537868 9834 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:32915
I20260806 02:17:00.538168 9910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:32915 every 8 connection(s)
I20260806 02:17:00.538833 9834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:17:00.541528 9911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:17:00.545158 9911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:17:00.546226 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 9834
I20260806 02:17:00.546350 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:17:00.546633 18713 external_mini_cluster.cc:1428] Setting key 028db85e659b3753bf53aeb5caa95519
Aug 06 02:17:00 dist-test-slave-6k22 krb5kdc[9818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.70.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.125@KRBTEST.COM" created.
I20260806 02:17:00.560182 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.547523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60382 (local address 127.18.70.126:32915)
0806 02:17:00.548133 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:00.548146 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:00.548194 (+ 48us) server_negotiation.cc:407] Connection header received
0806 02:17:00.549247 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:00.549251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:00.549420 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:00.549596 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:00.550408 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.551337 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.552350 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.552738 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.552922 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:00.555463 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:00.555468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:00.555479 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:00.555500 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:00.557119 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:00.557588 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:00.557592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:00.557593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:00.557706 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:00.558047 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:00.558050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:00.558052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:00.558419 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:00.558524 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:00.559003 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 02:17:00.559280 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":484,"spinlock_wait_cycles":65920,"thread_start_us":188,"threads_started":1}
I20260806 02:17:00.560179 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.547523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:32915 (local address 127.0.0.1:60382)
0806 02:17:00.548054 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.548107 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:00.548937 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:00.549598 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:00.549603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:00.550027 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:00.550269 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:00.550281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.551457 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.551461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:00.552234 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:00.552244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.552905 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.552909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:00.553021 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:00.553591 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:00.553608 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:00.555328 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:00.557233 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:00.557242 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:00.557250 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:00.557465 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:00.557798 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:00.557800 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:00.557802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:00.557926 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:00.558540 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:00.558550 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:00.558728 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 02:17:00.558920 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 02:17:00.559377 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":118,"thread_start_us":149,"threads_started":1}
I20260806 02:17:00.562700 9911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:17:00.563442 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.563114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:37237 (local address 127.0.0.1:50016)
0806 02:17:00.563251 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.563359 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:37237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 02:17:00.563674 9854 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.70.125:37237: connect: Connection refused (error 111)
I20260806 02:17:00.564574 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.564312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:37237 (local address 127.0.0.1:50022)
0806 02:17:00.564442 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.564500 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:37237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 02:17:00.564761 9911 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.125:37237: Network error: Client connection negotiation failed: client connection to 127.18.70.125:37237: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.125.keytab.
Entry for principal kudu/127.18.70.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.125.keytab.
I20260806 02:17:00.572382 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.125:37237
--webserver_interface=127.18.70.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.125.keytab
--principal=kudu/127.18.70.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:42147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:32915,127.18.70.125:37237,127.18.70.124:40673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:17:00.614045 9911 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } attempt: 1
I20260806 02:17:00.615051 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.614731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:37237 (local address 127.0.0.1:50038)
0806 02:17:00.614890 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.614960 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:37237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 02:17:00.615839 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.615608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:37237 (local address 127.0.0.1:50052)
0806 02:17:00.615701 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.615769 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:37237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 02:17:00.615965 9911 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.125:37237: Network error: Client connection negotiation failed: client connection to 127.18.70.125:37237: connect: Connection refused (error 111)
W20260806 02:17:00.683386 9918 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 02:17:00.683676 9918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:17:00.683748 9918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:17:00.683784 9918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 02:17:00.684121 9911 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } attempt: 2
I20260806 02:17:00.685019 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.684688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:37237 (local address 127.0.0.1:50056)
0806 02:17:00.684867 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.684930 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:37237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 02:17:00.685739 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.685515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:37237 (local address 127.0.0.1:50066)
0806 02:17:00.685612 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.685682 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:37237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 02:17:00.685933 9911 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.125:37237: Network error: Client connection negotiation failed: client connection to 127.18.70.125:37237: connect: Connection refused (error 111)
W20260806 02:17:00.688283 9918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:17:00.688331 9918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:17:00.688350 9918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:17:00.688367 9918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:17:00.688416 9918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:17:00.693463 9918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:42147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:32915,127.18.70.125:37237,127.18.70.124:40673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.125.keytab
--principal=kudu/127.18.70.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.125:37237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.70.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:17:00.694418 9918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:17:00.695401 9918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:17:00.703349 9931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:17:00.703333 9929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:17:00.703328 9928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:17:00.703811 9918 server_base.cc:1034] running on GCE node
I20260806 02:17:00.704284 9918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:17:00.705267 9918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:17:00.706494 9918 hybrid_clock.cc:648] HybridClock initialized: now 1785982620706453 us; error 36 us; skew 500 ppm
Aug 06 02:17:00 dist-test-slave-6k22 krb5kdc[9818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:17:00.709435 9918 init.cc:377] Logged in from keytab as kudu/127.18.70.125@KRBTEST.COM (short username kudu)
I20260806 02:17:00.710227 9918 webserver.cc:460] Webserver started at http://127.18.70.125:37873/ using document root <none> and password file <none>
I20260806 02:17:00.710752 9918 fs_manager.cc:359] Metadata directory not provided
I20260806 02:17:00.710809 9918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:17:00.711058 9918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:17:00.713097 9918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/data/instance:
uuid: "8e27f5d1b3d4462a95311392fd7f60f3"
format_stamp: "Formatted at 2026-08-06 02:17:00 on dist-test-slave-6k22"
server_key: "12e0d607bbde7d912152d4cae9807a0f"
server_key_iv: "04605c8642c7fca02250250dfbe0a943"
server_key_version: "encryptionkey@0"
I20260806 02:17:00.713649 9918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/wal/instance:
uuid: "8e27f5d1b3d4462a95311392fd7f60f3"
format_stamp: "Formatted at 2026-08-06 02:17:00 on dist-test-slave-6k22"
server_key: "12e0d607bbde7d912152d4cae9807a0f"
server_key_iv: "04605c8642c7fca02250250dfbe0a943"
server_key_version: "encryptionkey@0"
I20260806 02:17:00.717304 9918 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 02:17:00.719648 9938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:17:00.720844 9918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:17:00.720966 9918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/wal
uuid: "8e27f5d1b3d4462a95311392fd7f60f3"
format_stamp: "Formatted at 2026-08-06 02:17:00 on dist-test-slave-6k22"
server_key: "12e0d607bbde7d912152d4cae9807a0f"
server_key_iv: "04605c8642c7fca02250250dfbe0a943"
server_key_version: "encryptionkey@0"
I20260806 02:17:00.721069 9918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:17:00.750926 9918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:17:00.753703 9918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:17:00.753901 9918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:17:00.761071 9918 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.125:37237
I20260806 02:17:00.761350 9992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.125:37237 every 8 connection(s)
I20260806 02:17:00.762068 9918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/data/info.pb
I20260806 02:17:00.764895 9993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:17:00.768777 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 9918
I20260806 02:17:00.768914 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-1/wal/instance
I20260806 02:17:00.768621 9993 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:17:00.769177 18713 external_mini_cluster.cc:1428] Setting key 38cafc2d91f457bb0b78fee0c3aa5025
I20260806 02:17:00.777099 9911 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } attempt: 3
WARNING: no policy specified for kudu/127.18.70.124@KRBTEST.COM; defaulting to no policy
Aug 06 02:17:00 dist-test-slave-6k22 krb5kdc[9818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.125@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.124@KRBTEST.COM" created.
I20260806 02:17:00.783123 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.770553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60394 (local address 127.18.70.126:32915)
0806 02:17:00.770741 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:00.770746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:00.771593 (+ 847us) server_negotiation.cc:407] Connection header received
0806 02:17:00.772509 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:00.772515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:00.772573 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:00.772662 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:00.773731 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.774592 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.775686 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.776074 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.776193 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:00.778771 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:00.778777 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:00.778783 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:00.778812 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:00.780361 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:00.780892 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:00.780898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:00.780903 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:00.780980 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:00.781736 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:00.781741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:00.781742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:00.781993 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:00.782111 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:00.782824 (+ 713us) server_negotiation.cc:299] Negotiation successful
0806 02:17:00.782950 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
Aug 06 02:17:00 dist-test-slave-6k22 krb5kdc[9818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for kudu/127.18.70.125@KRBTEST.COM
I20260806 02:17:00.784539 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.770947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:32915 (local address 127.0.0.1:60394)
0806 02:17:00.771407 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.771459 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:00.772314 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:00.772812 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:00.772818 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:00.773132 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:00.773584 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:00.773597 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.774719 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.774721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:00.775574 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:00.775584 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.776218 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.776220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:00.776356 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:00.776895 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:00.776911 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:00.778645 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:00.780467 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:00.780473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:00.780488 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:00.780797 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:00.781423 (+ 626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:00.781426 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:00.781429 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:00.781615 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:00.782131 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:00.782140 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:00.782451 (+ 311us) client_negotiation.cc:769] Sending connection context
0806 02:17:00.783330 (+ 879us) client_negotiation.cc:240] Negotiation successful
0806 02:17:00.783672 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":133,"threads_started":1}
I20260806 02:17:00.785957 9993 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:17:00.788203 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.777483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:37237 (local address 127.0.0.1:50070)
0806 02:17:00.777677 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.777700 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:00.777769 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:00.778489 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:00.778492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:00.778629 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:00.778748 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:00.778755 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.779809 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.779815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:00.780363 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:00.780373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.781099 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.781103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:00.781222 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:00.781667 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:00.781688 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:00.783722 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:00.786493 (+ 2771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:00.786498 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:00.786502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:00.786775 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:00.787074 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:00.787077 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:00.787079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:00.787162 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:00.787661 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:00.787666 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:00.787777 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 02:17:00.787891 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 02:17:00.788020 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260806 02:17:00.788537 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.777591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50070 (local address 127.18.70.125:37237)
0806 02:17:00.777998 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:00.778016 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:00.778039 (+ 23us) server_negotiation.cc:407] Connection header received
0806 02:17:00.778076 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:00.778083 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:00.778274 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:00.778471 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:00.778865 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.779699 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.780496 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.780966 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.781111 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:00.783877 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:00.783890 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:00.783907 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:00.783937 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:00.786345 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:00.786894 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:00.786902 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:00.786907 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:00.786992 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:00.787264 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:00.787267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:00.787269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:00.787542 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:00.787699 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:00.787996 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 02:17:00.788123 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":102,"threads_started":1}
I20260806 02:17:00.790638 9911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:17:00.791361 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.791150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:41948)
0806 02:17:00.791249 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.791295 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 02:17:00.792021 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.791806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:41952)
0806 02:17:00.791937 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.791977 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260806 02:17:00.792222 9911 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:40673: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Aug 06 02:17:00 dist-test-slave-6k22 krb5kdc[9818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.125@KRBTEST.COM for kudu/127.18.70.125@KRBTEST.COM
I20260806 02:17:00.796080 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.786440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:37237 (local address 127.0.0.1:50084)
0806 02:17:00.786600 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.786617 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:00.786703 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:00.786973 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:00.786979 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:00.787156 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:00.787287 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:00.787295 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.788232 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.788235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:00.788887 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:00.788896 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.789838 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.789841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:00.789971 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:00.790387 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:00.790401 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:00.792907 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:00.794711 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:00.794713 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:00.794714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:00.794908 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:00.795201 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:00.795206 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:00.795208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:00.795312 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:00.795724 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:00.795727 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:00.795804 (+ 77us) client_negotiation.cc:769] Sending connection context
0806 02:17:00.795884 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 02:17:00.795983 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":18}
I20260806 02:17:00.796277 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.786446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50084 (local address 127.18.70.125:37237)
0806 02:17:00.786765 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:00.786776 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:00.786787 (+ 11us) server_negotiation.cc:407] Connection header received
0806 02:17:00.786813 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:00.786816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:00.786854 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:00.786948 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:00.787422 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.788110 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.788993 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.789726 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.789841 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:00.793053 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:00.793056 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:00.793059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:00.793095 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:00.794600 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:00.795000 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:00.795002 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:00.795003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:00.795104 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:00.795403 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:00.795405 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:00.795406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:00.795602 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:00.795729 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:00.795924 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 02:17:00.796043 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":252,"thread_start_us":110,"threads_started":1}
I20260806 02:17:00.796864 9993 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:17:00.797415 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.797215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:41956)
0806 02:17:00.797306 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.797369 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 02:17:00.797617 9940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.124.keytab.
Entry for principal kudu/127.18.70.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.124.keytab.
I20260806 02:17:00.798460 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.798246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:41970)
0806 02:17:00.798352 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.798415 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 02:17:00.798655 9993 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:40673: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
I20260806 02:17:00.799180 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.124:40673
--webserver_interface=127.18.70.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.124.keytab
--principal=kudu/127.18.70.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:42147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:32915,127.18.70.125:37237,127.18.70.124:40673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:17:00.823428 9911 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } attempt: 1
I20260806 02:17:00.824326 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.824039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:41982)
0806 02:17:00.824187 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.824248 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 02:17:00.825111 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.824824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:41986)
0806 02:17:00.824937 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.825021 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 02:17:00.825260 9911 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:40673: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
I20260806 02:17:00.847882 9993 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } attempt: 1
I20260806 02:17:00.848730 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.848428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:41992)
0806 02:17:00.848597 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.848654 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 02:17:00.849473 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.849244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:42002)
0806 02:17:00.849369 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.849417 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 02:17:00.849601 9993 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:40673: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
I20260806 02:17:00.900411 9911 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } attempt: 2
I20260806 02:17:00.901249 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.900946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:42010)
0806 02:17:00.901102 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.901164 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 02:17:00.901894 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.901688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:42016)
0806 02:17:00.901797 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.901845 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 02:17:00.902015 9911 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:40673: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
I20260806 02:17:00.917747 9993 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } attempt: 2
I20260806 02:17:00.918561 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.918289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:42026)
0806 02:17:00.918437 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.918492 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 02:17:00.919256 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.919048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:42036)
0806 02:17:00.919151 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.919203 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 02:17:00.919381 9993 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:40673: Network error: Client connection negotiation failed: client connection to 127.18.70.124:40673: connect: Connection refused (error 111)
W20260806 02:17:00.920804 10001 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 02:17:00.921161 10001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:17:00.921231 10001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:17:00.921317 10001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:17:00.925828 10001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:17:00.925910 10001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:17:00.925966 10001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:17:00.926026 10001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:17:00.926088 10001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:17:00.931512 10001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:42147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:32915,127.18.70.125:37237,127.18.70.124:40673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.124.keytab
--principal=kudu/127.18.70.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.124:40673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.70.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:17:00.932598 10001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:17:00.933601 10001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:17:00.942020 10010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:17:00.942029 10007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:17:00.942039 10006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:17:00.942389 10001 server_base.cc:1034] running on GCE node
I20260806 02:17:00.942878 10001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:17:00.943956 10001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:17:00.945108 10001 hybrid_clock.cc:648] HybridClock initialized: now 1785982620945091 us; error 24 us; skew 500 ppm
Aug 06 02:17:00 dist-test-slave-6k22 krb5kdc[9818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:17:00.947718 10001 init.cc:377] Logged in from keytab as kudu/127.18.70.124@KRBTEST.COM (short username kudu)
I20260806 02:17:00.948535 10001 webserver.cc:460] Webserver started at http://127.18.70.124:44295/ using document root <none> and password file <none>
I20260806 02:17:00.949062 10001 fs_manager.cc:359] Metadata directory not provided
I20260806 02:17:00.949120 10001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:17:00.949347 10001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:17:00.951296 10001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/data/instance:
uuid: "31badfaba05e4e8ca79761bbfd4e5dc4"
format_stamp: "Formatted at 2026-08-06 02:17:00 on dist-test-slave-6k22"
server_key: "d69c699e5e5c6a9520be2eb54a48491b"
server_key_iv: "fd8c4949cb3772642f167ffe500713b8"
server_key_version: "encryptionkey@0"
I20260806 02:17:00.951870 10001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/wal/instance:
uuid: "31badfaba05e4e8ca79761bbfd4e5dc4"
format_stamp: "Formatted at 2026-08-06 02:17:00 on dist-test-slave-6k22"
server_key: "d69c699e5e5c6a9520be2eb54a48491b"
server_key_iv: "fd8c4949cb3772642f167ffe500713b8"
server_key_version: "encryptionkey@0"
I20260806 02:17:00.955346 10001 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 02:17:00.957643 10017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:17:00.958812 10001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:17:00.958941 10001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/wal
uuid: "31badfaba05e4e8ca79761bbfd4e5dc4"
format_stamp: "Formatted at 2026-08-06 02:17:00 on dist-test-slave-6k22"
server_key: "d69c699e5e5c6a9520be2eb54a48491b"
server_key_iv: "fd8c4949cb3772642f167ffe500713b8"
server_key_version: "encryptionkey@0"
I20260806 02:17:00.959041 10001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:17:00.973556 10001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:17:00.976408 10001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:17:00.976615 10001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:17:00.983321 10001 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.124:40673
I20260806 02:17:00.983652 10073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.124:40673 every 8 connection(s)
I20260806 02:17:00.984360 10001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/data/info.pb
I20260806 02:17:00.985410 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 10001
I20260806 02:17:00.985526 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785982339772911-18713-0/minicluster-data/master-2/wal/instance
I20260806 02:17:00.985833 18713 external_mini_cluster.cc:1428] Setting key fcb643b4747640bf0a94049f60626331
I20260806 02:17:00.987686 10074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:17:00.992168 10074 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 02:17:00 dist-test-slave-6k22 krb5kdc[9818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.124@KRBTEST.COM
Aug 06 02:17:00 dist-test-slave-6k22 krb5kdc[9818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.124@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:17:01.000425 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.987102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42038 (local address 127.18.70.124:40673)
0806 02:17:00.987529 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:00.987548 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:00.987612 (+ 64us) server_negotiation.cc:407] Connection header received
0806 02:17:00.988427 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:00.988449 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:00.988838 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:00.989184 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:00.989801 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.991280 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.992351 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.992986 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.993137 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:00.995822 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:00.995827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:00.995835 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:00.995858 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:00.997442 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:00.997947 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:00.997952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:00.997956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:00.998061 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:00.998391 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:00.998394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:00.998395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:00.998893 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:00.999102 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:00.999363 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 02:17:00.999674 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":132,"threads_started":1}
I20260806 02:17:01.001164 9911 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } attempt: 3
I20260806 02:17:01.003546 10079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.994038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:32915 (local address 127.0.0.1:60402)
0806 02:17:00.994391 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:00.994426 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:00.994555 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:00.994832 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:00.994837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:00.995128 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:00.995284 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:00.995295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.996169 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.996171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:00.996768 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:00.996776 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.997214 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.997218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:00.997328 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:00.997753 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:00.997771 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:00.999363 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:01.001584 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:01.001596 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:01.001601 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:01.001848 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:01.002127 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:01.002132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:01.002134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:01.002222 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:01.002954 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:01.002960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:01.003064 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 02:17:01.003197 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 02:17:01.003348 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":143,"threads_started":1}
I20260806 02:17:01.003603 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:00.994070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60402 (local address 127.18.70.126:32915)
0806 02:17:00.994183 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:00.994191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:00.994504 (+ 313us) server_negotiation.cc:407] Connection header received
0806 02:17:00.994657 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:00.994660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:00.994713 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:00.994787 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:00.995423 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.996076 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.996889 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:00.997109 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:00.997220 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:00.999571 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:00.999578 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:00.999581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:00.999608 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:01.001415 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.001950 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.001954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.001956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.002035 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.002569 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.002573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.002574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.002808 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:01.002902 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:01.003270 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 02:17:01.003391 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 02:17:01.004817 10074 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 02:17:01 dist-test-slave-6k22 krb5kdc[9818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for kudu/127.18.70.124@KRBTEST.COM
Aug 06 02:17:01 dist-test-slave-6k22 krb5kdc[9818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
Aug 06 02:17:01 dist-test-slave-6k22 krb5kdc[9818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.124@KRBTEST.COM for kudu/127.18.70.125@KRBTEST.COM
I20260806 02:17:01.010583 9993 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } attempt: 3
I20260806 02:17:01.011716 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:01.001569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:42042)
0806 02:17:01.001726 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:01.001742 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:01.001816 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:01.002160 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:01.002163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:01.002350 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:01.002445 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:01.002451 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.003456 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.003459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:01.004236 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:01.004246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.004804 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.004807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:01.004942 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:01.005517 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:01.005531 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:01.007957 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:01.010055 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:01.010059 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:01.010061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:01.010305 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:01.010630 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:01.010633 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:01.010635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:01.010732 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:01.011178 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:01.011181 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:01.011274 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 02:17:01.011384 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 02:17:01.011511 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":23}
I20260806 02:17:01.011745 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:01.001604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42042 (local address 127.18.70.124:40673)
0806 02:17:01.001954 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.001958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.001970 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:17:01.002000 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.002002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.002037 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.002559 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.002648 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.003333 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.004379 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.004695 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.004791 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:01.008087 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:01.008091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:01.008093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:01.008115 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:01.009885 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.010394 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.010397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.010398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.010484 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.010856 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.010862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.010867 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.011090 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:01.011182 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:01.011447 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 02:17:01.011596 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282}
I20260806 02:17:01.013410 10080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:01.002772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60408 (local address 127.18.70.126:32915)
0806 02:17:01.003088 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.003097 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.003108 (+ 11us) server_negotiation.cc:407] Connection header received
0806 02:17:01.003137 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.003139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.003194 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.004084 (+ 890us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.004205 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.004923 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.006000 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.006278 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.006354 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:01.009119 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:01.009122 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:01.009123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:01.009143 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:01.011115 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.011622 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.011626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.011628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.011730 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.012061 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.012067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.012071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.012289 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:01.012479 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:01.013091 (+ 612us) server_negotiation.cc:299] Negotiation successful
0806 02:17:01.013219 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":246,"thread_start_us":91,"threads_started":1}
I20260806 02:17:01.015417 9911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2: Bootstrap starting.
I20260806 02:17:01.015673 10079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:01.005292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:37237 (local address 127.0.0.1:50086)
0806 02:17:01.005459 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:01.005471 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:01.005541 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:01.005766 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:01.005769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:01.005925 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:01.006002 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:01.006006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.006912 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.006915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:01.008092 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:01.008104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.008727 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.008732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:01.008852 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:01.009381 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:01.009400 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:01.011669 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:01.013844 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:01.013847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:01.013849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:01.014136 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:01.014449 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:01.014452 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:01.014454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:01.014544 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:01.015122 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:01.015125 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:01.015210 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 02:17:01.015315 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 02:17:01.015444 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260806 02:17:01.016155 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:01.005381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50086 (local address 127.18.70.125:37237)
0806 02:17:01.005505 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.005510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.005524 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:17:01.005622 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.005625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.005672 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.005752 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.006119 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.006794 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.008228 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.008610 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.008731 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:01.011826 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:01.011832 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:01.011836 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:01.011861 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:01.013719 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.014262 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.014267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.014270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.014354 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.014669 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.014673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.014674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.014950 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:01.015032 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:01.015860 (+ 828us) server_negotiation.cc:299] Negotiation successful
0806 02:17:01.015986 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":49}
Aug 06 02:17:01 dist-test-slave-6k22 krb5kdc[9818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.125@KRBTEST.COM for kudu/127.18.70.124@KRBTEST.COM
I20260806 02:17:01.018373 9911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2: Neither blocks nor log segments found. Creating new log.
I20260806 02:17:01.019344 9911 log.cc:824] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2: Log is configured to *not* fsync() on all Append() calls
Aug 06 02:17:01 dist-test-slave-6k22 krb5kdc[9818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.125@KRBTEST.COM
I20260806 02:17:01.020521 10074 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:17:01.022215 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:01.010951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:42054)
0806 02:17:01.011077 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:01.011091 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:01.011170 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:01.012374 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:01.012378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:01.012605 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:01.012704 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:01.012711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.014807 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.014810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:01.015605 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:01.015616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.016219 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.016223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:01.016297 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:01.016857 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:01.016873 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:01.018297 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:01.020046 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:01.020049 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:01.020052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:01.020280 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:01.020609 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:01.020612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:01.020614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:01.020711 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:01.021418 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:01.021421 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:01.021520 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 02:17:01.021891 (+ 371us) client_negotiation.cc:240] Negotiation successful
0806 02:17:01.022035 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":18}
I20260806 02:17:01.022446 9911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2: No bootstrap required, opened a new log
I20260806 02:17:01.022679 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:01.011328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42054 (local address 127.18.70.124:40673)
0806 02:17:01.011910 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.011916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.011931 (+ 15us) server_negotiation.cc:407] Connection header received
0806 02:17:01.011962 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.011965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.011997 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.013073 (+ 1076us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.013155 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.014200 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.015773 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.016088 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.017449 (+ 1361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:01.018422 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:01.018426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:01.018428 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:01.018451 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:01.019942 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.020415 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.020418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.020420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.020502 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.020832 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.020837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.020842 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.021304 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:01.022374 (+ 1070us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:01.022444 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 02:17:01.022538 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":106,"threads_started":1}
I20260806 02:17:01.024757 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:01.014697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50098 (local address 127.18.70.125:37237)
0806 02:17:01.014795 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.014799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.014811 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:17:01.014917 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.014921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.014962 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.015010 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.015942 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.016562 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.017291 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.017627 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.017763 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:01.020763 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:01.020769 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:01.020774 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:01.020798 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:01.022460 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.022965 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.022968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.022970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.023055 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.023343 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.023346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.023348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.023535 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:01.023755 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:01.024504 (+ 749us) server_negotiation.cc:299] Negotiation successful
0806 02:17:01.024612 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 02:17:01 dist-test-slave-6k22 krb5kdc[9818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.124@KRBTEST.COM for kudu/127.18.70.124@KRBTEST.COM
I20260806 02:17:01.026592 9911 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } }
I20260806 02:17:01.026815 9911 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5a5297f61f64240a87031da280c33f2, State: Initialized, Role: FOLLOWER
I20260806 02:17:01.027024 9993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3: Bootstrap starting.
I20260806 02:17:01.027137 9911 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } }
I20260806 02:17:01.027619 10084 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } } }
I20260806 02:17:01.027813 10084 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:17:01.028543 9911 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:17:01.029678 9993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3: Neither blocks nor log segments found. Creating new log.
I20260806 02:17:01.029870 10081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:01.021108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42062 (local address 127.18.70.124:40673)
0806 02:17:01.021430 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.021441 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.021451 (+ 10us) server_negotiation.cc:407] Connection header received
0806 02:17:01.021478 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.021480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.021514 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.021593 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.022067 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.022798 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.023778 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.024132 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.024230 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:01.026304 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:01.026308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:01.026310 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:01.026333 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:01.028049 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.028531 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.028534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.028536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.028618 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.028917 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.028920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.028922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.029115 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:01.029213 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:01.029571 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 02:17:01.029701 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266}
I20260806 02:17:01.030535 10079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:01.020916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:40673 (local address 127.0.0.1:42062)
0806 02:17:01.021224 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:01.021239 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:01.021305 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:01.021698 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:01.021702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:01.021876 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:01.021955 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:01.021962 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.022895 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.022898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:01.023651 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:01.023662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.024327 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.024333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:01.024418 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:01.024862 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:01.024894 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:01.026168 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:01.028175 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:01.028178 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:01.028180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:01.028428 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:01.028701 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:01.028704 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:01.028706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:01.028793 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:01.029226 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:01.029233 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:01.029339 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 02:17:01.030184 (+ 845us) client_negotiation.cc:240] Negotiation successful
0806 02:17:01.030351 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224}
I20260806 02:17:01.030524 9993 log.cc:824] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3: Log is configured to *not* fsync() on all Append() calls
W20260806 02:17:01.032670 9856 tablet.cc:2369] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:17:01.033486 10074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4: Bootstrap starting.
W20260806 02:17:01.035288 10095 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:17:01.035364 10095 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:17:01.035609 10074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4: Neither blocks nor log segments found. Creating new log.
I20260806 02:17:01.036440 10074 log.cc:824] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4: Log is configured to *not* fsync() on all Append() calls
I20260806 02:17:01.037586 9993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3: No bootstrap required, opened a new log
I20260806 02:17:01.034732 9834 master_runner.cc:186] Error getting master registration for 127.18.70.125:37237: OK, instance_id { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" instance_seqno: 1785982620758975 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 02:17:01.037827 9834 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:17:01.037935 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:17:01.039345 9880 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f5a5297f61f64240a87031da280c33f2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60408
I20260806 02:17:01.039783 9880 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 02:17:01.039901 9880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:17:01.040546 10074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4: No bootstrap required, opened a new log
I20260806 02:17:01.041074 9880 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } }
I20260806 02:17:01.041603 9993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } }
I20260806 02:17:01.041810 9993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e27f5d1b3d4462a95311392fd7f60f3, State: Initialized, Role: FOLLOWER
I20260806 02:17:01.041985 9880 leader_election.cc:288] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [CANDIDATE]: Term 1 election: Requested vote from peers 8e27f5d1b3d4462a95311392fd7f60f3 (127.18.70.125:37237), 31badfaba05e4e8ca79761bbfd4e5dc4 (127.18.70.124:40673)
I20260806 02:17:01.042110 9993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } }
I20260806 02:17:01.043540 9993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:17:01.044780 10074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } }
I20260806 02:17:01.045018 10074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31badfaba05e4e8ca79761bbfd4e5dc4, State: Initialized, Role: FOLLOWER
I20260806 02:17:01.045331 10074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } }
I20260806 02:17:01.045998 9965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5a5297f61f64240a87031da280c33f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8e27f5d1b3d4462a95311392fd7f60f3"
I20260806 02:17:01.046316 9965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:17:01.046320 10116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } } }
I20260806 02:17:01.046458 10116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:17:01.047232 10074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:17:01.047793 9965 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5a5297f61f64240a87031da280c33f2 in term 1.
I20260806 02:17:01.048283 9854 leader_election.cc:302] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e27f5d1b3d4462a95311392fd7f60f3, f5a5297f61f64240a87031da280c33f2; no voters:
I20260806 02:17:01.048617 10084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:17:01.048862 10084 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [term 1 LEADER]: Becoming Leader. State: Replica: f5a5297f61f64240a87031da280c33f2, State: Running, Role: LEADER
I20260806 02:17:01.049190 10084 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } }
I20260806 02:17:01.051893 10124 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5a5297f61f64240a87031da280c33f2. Latest consensus state: current_term: 1 leader_uuid: "f5a5297f61f64240a87031da280c33f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } } }
I20260806 02:17:01.052032 10124 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [sys.catalog]: This master's current role is: LEADER
I20260806 02:17:01.052191 10099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } } }
I20260806 02:17:01.052348 10099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:17:01.052466 10046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5a5297f61f64240a87031da280c33f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4"
I20260806 02:17:01.052719 10046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4 [term 0 FOLLOWER]: Advancing to term 1
W20260806 02:17:01.053746 10130 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:17:01.053818 10130 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:17:01.054087 10046 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5a5297f61f64240a87031da280c33f2 in term 1.
W20260806 02:17:01.055768 10117 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:17:01.055855 10117 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:17:01.061734 9918 master_runner.cc:263] No leader master found from master 8e27f5d1b3d4462a95311392fd7f60f3
I20260806 02:17:01.061802 9918 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:17:01.067883 10137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:17:01.068998 10137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:17:01.069887 10081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:01.050132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42068 (local address 127.18.70.124:40673)
0806 02:17:01.050249 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.050254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.054161 (+ 3907us) server_negotiation.cc:407] Connection header received
0806 02:17:01.054382 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.054385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.054439 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.054497 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.057914 (+ 3417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.058745 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.060053 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.060405 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.060494 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:01.062793 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:01.062797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:01.062803 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:01.062832 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:01.065604 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.066483 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.066487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.066489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.066590 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.067837 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.067841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.067842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.068081 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:01.068256 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:01.069584 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0806 02:17:01.069723 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 02:17:01.072238 10080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:01.054886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60410 (local address 127.18.70.126:32915)
0806 02:17:01.055129 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.055133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.056755 (+ 1622us) server_negotiation.cc:407] Connection header received
0806 02:17:01.056802 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.056805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.056851 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.056950 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.063333 (+ 6383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.064243 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.065518 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.065924 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.066000 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:01.067872 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:01.067876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:01.067879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:01.067907 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:01.069886 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.070676 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.070680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.070682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.070774 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.071089 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.071093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.071095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.071381 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:01.071528 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:01.071933 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 02:17:01.072063 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260806 02:17:01.077711 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:01.051677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50110 (local address 127.18.70.125:37237)
0806 02:17:01.051798 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.051803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.058221 (+ 6418us) server_negotiation.cc:407] Connection header received
0806 02:17:01.058271 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.058275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.058320 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.058425 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.066943 (+ 8518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.067679 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.071500 (+ 3821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:01.071920 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:01.072026 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:01.074008 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:01.074012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:01.074015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:01.074045 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:01.075831 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.076307 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.076311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.076313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.076410 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:01.076742 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:01.076746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:01.076748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:01.076975 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:01.077126 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:01.077395 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 02:17:01.077519 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 02:17:01.085480 10046 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4 [term 1 FOLLOWER]: Refusing update from remote peer f5a5297f61f64240a87031da280c33f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:17:01.086297 10124 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:17:01.090574 10116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5a5297f61f64240a87031da280c33f2. Latest consensus state: current_term: 1 leader_uuid: "f5a5297f61f64240a87031da280c33f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } } }
I20260806 02:17:01.090708 10116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:17:01.094034 9965 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3 [term 1 FOLLOWER]: Refusing update from remote peer f5a5297f61f64240a87031da280c33f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:17:01.094646 10124 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:17:01.098869 10099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5a5297f61f64240a87031da280c33f2. Latest consensus state: current_term: 1 leader_uuid: "f5a5297f61f64240a87031da280c33f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } } }
I20260806 02:17:01.098994 10099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:17:01.108553 10124 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5a5297f61f64240a87031da280c33f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } } }
I20260806 02:17:01.108721 10124 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [sys.catalog]: This master's current role is: LEADER
I20260806 02:17:01.112021 10099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5a5297f61f64240a87031da280c33f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } } }
I20260806 02:17:01.112125 10099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:17:01.112739 10139 mvcc.cc:205] Tried to move back new op lower bound from 7315384815949869056 to 7315384815821082624. Current Snapshot: MvccSnapshot[applied={T|T < 7315384815949869056}]
I20260806 02:17:01.113688 10084 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5a5297f61f64240a87031da280c33f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } } }
I20260806 02:17:01.113807 10084 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2 [sys.catalog]: This master's current role is: LEADER
I20260806 02:17:01.114935 10116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5a5297f61f64240a87031da280c33f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a5297f61f64240a87031da280c33f2" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 32915 } } peers { permanent_uuid: "8e27f5d1b3d4462a95311392fd7f60f3" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 37237 } } peers { permanent_uuid: "31badfaba05e4e8ca79761bbfd4e5dc4" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 40673 } } }
I20260806 02:17:01.115049 10116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:17:01.115906 10142 mvcc.cc:205] Tried to move back new op lower bound from 7315384815949869056 to 7315384815821082624. Current Snapshot: MvccSnapshot[applied={T|T < 7315384815949869056}]
I20260806 02:17:01.116117 10140 mvcc.cc:205] Tried to move back new op lower bound from 7315384815949869056 to 7315384815821082624. Current Snapshot: MvccSnapshot[applied={T|T < 7315384815949869056}]
I20260806 02:17:01.123028 10137 catalog_manager.cc:1347] Generated new cluster ID: 11135295509a466a93852142cc52a73e
I20260806 02:17:01.123106 10137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:17:01.151798 10137 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:17:01.153105 10137 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:17:01.171872 10137 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5a5297f61f64240a87031da280c33f2: Generated new TSK 0
I20260806 02:17:01.172665 10137 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 02:17:01.208452 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:01.205361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50126 (local address 127.18.70.125:37237)
0806 02:17:01.205482 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.205488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.207364 (+ 1876us) server_negotiation.cc:407] Connection header received
0806 02:17:01.207563 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.207581 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.207628 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.207694 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.208326 (+ 632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50126: BlockingRecv error: recv got EOF from 127.0.0.1:50126 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 02:17:01.210104 10080 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:01.206145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60422 (local address 127.18.70.126:32915)
0806 02:17:01.206262 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.206267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.208911 (+ 2644us) server_negotiation.cc:407] Connection header received
0806 02:17:01.209380 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.209384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.209433 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.209485 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.209992 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60422: BlockingRecv error: recv got EOF from 127.0.0.1:60422 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 02:17:01.211153 10081 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:01.206842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42078 (local address 127.18.70.124:40673)
0806 02:17:01.206967 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.206973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.209114 (+ 2141us) server_negotiation.cc:407] Connection header received
0806 02:17:01.210353 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.210356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.210409 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.210462 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.211036 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42078: BlockingRecv error: recv got EOF from 127.0.0.1:42078 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 02:17:01.215155 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:01.213545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50128 (local address 127.18.70.125:37237)
0806 02:17:01.213664 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.213668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.214192 (+ 524us) server_negotiation.cc:407] Connection header received
0806 02:17:01.214454 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.214457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.214522 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.214572 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.215072 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50128: BlockingRecv error: recv got EOF from 127.0.0.1:50128 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 02:17:01.220403 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:01.218809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50132 (local address 127.18.70.125:37237)
0806 02:17:01.218903 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.218908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.219471 (+ 563us) server_negotiation.cc:407] Connection header received
0806 02:17:01.219691 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.219694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.219741 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.219798 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.220317 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50132: BlockingRecv error: recv got EOF from 127.0.0.1:50132 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 02:17:01.226215 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:01.224705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50138 (local address 127.18.70.125:37237)
0806 02:17:01.224816 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.224820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.225329 (+ 509us) server_negotiation.cc:407] Connection header received
0806 02:17:01.225554 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.225557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.225600 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.225656 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.226120 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50138: BlockingRecv error: recv got EOF from 127.0.0.1:50138 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 02:17:01.233624 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:01.232095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50148 (local address 127.18.70.125:37237)
0806 02:17:01.232215 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.232219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.232765 (+ 546us) server_negotiation.cc:407] Connection header received
0806 02:17:01.232952 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.232956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.233005 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.233057 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.233542 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50148: BlockingRecv error: recv got EOF from 127.0.0.1:50148 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 02:17:01.245486 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:01.243784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50154 (local address 127.18.70.125:37237)
0806 02:17:01.243891 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.243895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.244441 (+ 546us) server_negotiation.cc:407] Connection header received
0806 02:17:01.244778 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.244782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.244827 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.244874 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.245375 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50154: BlockingRecv error: recv got EOF from 127.0.0.1:50154 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 02:17:01.265185 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:01.263808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50168 (local address 127.18.70.125:37237)
0806 02:17:01.263958 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.263963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.264282 (+ 319us) server_negotiation.cc:407] Connection header received
0806 02:17:01.264435 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.264438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.264491 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.264574 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.265074 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50168: BlockingRecv error: recv got EOF from 127.0.0.1:50168 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 02:17:01.301354 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:01.299900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50176 (local address 127.18.70.125:37237)
0806 02:17:01.300094 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.300100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.300242 (+ 142us) server_negotiation.cc:407] Connection header received
0806 02:17:01.300434 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.300438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.300499 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.300598 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.301213 (+ 615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50176: BlockingRecv error: recv got EOF from 127.0.0.1:50176 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260806 02:17:01.368873 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:01.367628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50184 (local address 127.18.70.125:37237)
0806 02:17:01.367782 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.367787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.367920 (+ 133us) server_negotiation.cc:407] Connection header received
0806 02:17:01.368110 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.368114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.368182 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.368284 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.368735 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50184: BlockingRecv error: recv got EOF from 127.0.0.1:50184 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 02:17:01.500519 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:01.499240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50198 (local address 127.18.70.125:37237)
0806 02:17:01.499397 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.499402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.499595 (+ 193us) server_negotiation.cc:407] Connection header received
0806 02:17:01.499787 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.499790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.499848 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.499939 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.500390 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50198: BlockingRecv error: recv got EOF from 127.0.0.1:50198 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 02:17:01.759832 9997 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:01.758631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50208 (local address 127.18.70.125:37237)
0806 02:17:01.758788 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:01.758793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:01.758904 (+ 111us) server_negotiation.cc:407] Connection header received
0806 02:17:01.759079 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:01.759082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:01.759138 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:01.759228 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:01.759716 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50208: BlockingRecv error: recv got EOF from 127.0.0.1:50208 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 02:17:02.055210 10130 catalog_manager.cc:1259] Loaded cluster ID: 11135295509a466a93852142cc52a73e
I20260806 02:17:02.055315 10130 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4: loading cluster ID for follower catalog manager: success
I20260806 02:17:02.057093 10117 catalog_manager.cc:1259] Loaded cluster ID: 11135295509a466a93852142cc52a73e
I20260806 02:17:02.057166 10117 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3: loading cluster ID for follower catalog manager: success
I20260806 02:17:02.057544 10130 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4: acquiring CA information for follower catalog manager: success
I20260806 02:17:02.058952 10130 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 31badfaba05e4e8ca79761bbfd4e5dc4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:17:02.059175 10117 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3: acquiring CA information for follower catalog manager: success
I20260806 02:17:02.060166 10117 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8e27f5d1b3d4462a95311392fd7f60f3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:17:02.277935 10216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:02.273820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50220 (local address 127.18.70.125:37237)
0806 02:17:02.274114 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:02.274118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:02.274133 (+ 15us) server_negotiation.cc:407] Connection header received
0806 02:17:02.274262 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:02.274265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:02.274396 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:02.274497 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:17:02.274738 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:02.275245 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:02.276162 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:02.276411 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:02.276560 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:02.276840 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:17:02.277426 (+ 586us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:17:02.277547 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:02.277585 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 02:17:02.277728 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":94,"threads_started":1}
I20260806 02:17:02.284198 10222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:02.280973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42086 (local address 127.18.70.124:40673)
0806 02:17:02.281223 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:02.281226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:02.281240 (+ 14us) server_negotiation.cc:407] Connection header received
0806 02:17:02.281318 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:02.281321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:02.281429 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:02.281523 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:17:02.281710 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:02.282195 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:02.282908 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:02.283100 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:02.283173 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:02.283394 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:17:02.283838 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:17:02.283925 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:02.283981 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 02:17:02.284021 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":66,"threads_started":1}
I20260806 02:17:02.290566 10222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:02.287124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42088 (local address 127.18.70.124:40673)
0806 02:17:02.287226 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:02.287229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:02.287347 (+ 118us) server_negotiation.cc:407] Connection header received
0806 02:17:02.287409 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:02.287414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:02.287457 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:02.287533 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:17:02.287892 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:02.288651 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:02.289629 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:02.289842 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:02.289935 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:02.290162 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:17:02.290224 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:17:02.290282 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:02.290397 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 02:17:02.290452 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 02:17:02.291359 10216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:02.286986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.18.70.125:37237)
0806 02:17:02.287095 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:02.287098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:02.287878 (+ 780us) server_negotiation.cc:407] Connection header received
0806 02:17:02.288050 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:02.288053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:02.288090 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:02.288164 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:17:02.288426 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:02.289032 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:02.290079 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:02.290396 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:02.290483 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:02.290820 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:17:02.290922 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:17:02.290968 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:02.291140 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 02:17:02.291218 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 02:17:02.291967 10229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:02.286772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.18.70.126:32915)
0806 02:17:02.287168 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:02.287172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:02.287184 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:17:02.287279 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:02.287281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:02.287429 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:02.287545 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:17:02.288073 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:02.288821 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:02.290023 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:02.290303 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:02.290396 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:02.291104 (+ 708us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:17:02.291566 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:17:02.291678 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:02.291767 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 02:17:02.291828 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":93,"threads_started":1}
I20260806 02:17:02.294799 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 9834
I20260806 02:17:02.303828 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 9918
I20260806 02:17:02.310539 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 10001
2026-08-06T02:17:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3131 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 02:17:02 dist-test-slave-6k22 krb5kdc[10233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 02:17:02 dist-test-slave-6k22 krb5kdc[10233](info): set up 2 sockets
Aug 06 02:17:02 dist-test-slave-6k22 krb5kdc[10233](info): commencing operation
krb5kdc: starting...
W20260806 02:17:03.369585 18713 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/test-user.keytab.
Aug 06 02:17:03 dist-test-slave-6k22 krb5kdc[10233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T02:17:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T02:17:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.70.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
Entry for principal kudu/127.18.70.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab.
I20260806 02:17:03.492013 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:39099
--webserver_interface=127.18.70.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:46427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:39099,127.18.70.125:40561,127.18.70.124:39837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 02:17:03.601503 10248 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 02:17:03.601759 10248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:17:03.601783 10248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:17:03.601817 10248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:17:03.605520 10248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:17:03.605571 10248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:17:03.605588 10248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:17:03.605603 10248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:17:03.605618 10248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:17:03.610575 10248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:46427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:39099,127.18.70.125:40561,127.18.70.124:39837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.126.keytab
--principal=kudu/127.18.70.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.126:39099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.70.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:17:03.611505 10248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:17:03.612545 10248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:17:03.620872 10253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:17:03.620949 10254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:17:03.620886 10256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:17:03.620956 10248 server_base.cc:1034] running on GCE node
I20260806 02:17:03.621628 10248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:17:03.622740 10248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:17:03.623898 10248 hybrid_clock.cc:648] HybridClock initialized: now 1785982623623867 us; error 41 us; skew 500 ppm
Aug 06 02:17:03 dist-test-slave-6k22 krb5kdc[10233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982623, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:17:03.626693 10248 init.cc:377] Logged in from keytab as kudu/127.18.70.126@KRBTEST.COM (short username kudu)
I20260806 02:17:03.627631 10248 webserver.cc:460] Webserver started at http://127.18.70.126:45645/ using document root <none> and password file <none>
I20260806 02:17:03.628176 10248 fs_manager.cc:359] Metadata directory not provided
I20260806 02:17:03.628265 10248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:17:03.628486 10248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:17:03.630447 10248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/data/instance:
uuid: "71b9558f5262453aaf5b11023fdc0251"
format_stamp: "Formatted at 2026-08-06 02:17:03 on dist-test-slave-6k22"
server_key: "ccf193dc02ada33b0c3e6d0eea8acdb4"
server_key_iv: "07ed5f155ef5d0f0ab1ee8e9558f8ad9"
server_key_version: "encryptionkey@0"
I20260806 02:17:03.631008 10248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/wal/instance:
uuid: "71b9558f5262453aaf5b11023fdc0251"
format_stamp: "Formatted at 2026-08-06 02:17:03 on dist-test-slave-6k22"
server_key: "ccf193dc02ada33b0c3e6d0eea8acdb4"
server_key_iv: "07ed5f155ef5d0f0ab1ee8e9558f8ad9"
server_key_version: "encryptionkey@0"
I20260806 02:17:03.634810 10248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 02:17:03.637305 10263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:17:03.638514 10248 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 02:17:03.638657 10248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/wal
uuid: "71b9558f5262453aaf5b11023fdc0251"
format_stamp: "Formatted at 2026-08-06 02:17:03 on dist-test-slave-6k22"
server_key: "ccf193dc02ada33b0c3e6d0eea8acdb4"
server_key_iv: "07ed5f155ef5d0f0ab1ee8e9558f8ad9"
server_key_version: "encryptionkey@0"
I20260806 02:17:03.638770 10248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:17:03.654114 10248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:17:03.657347 10248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:17:03.657555 10248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:17:03.664517 10248 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.126:39099
I20260806 02:17:03.664780 10318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.126:39099 every 8 connection(s)
I20260806 02:17:03.665464 10248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/data/info.pb
I20260806 02:17:03.668011 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 10248
I20260806 02:17:03.668133 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-0/wal/instance
I20260806 02:17:03.668366 10319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:17:03.668402 18713 external_mini_cluster.cc:1428] Setting key e6dbb9f62887891126144724c0a0e79e
I20260806 02:17:03.673326 10319 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.70.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.125@KRBTEST.COM" created.
Aug 06 02:17:03 dist-test-slave-6k22 krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982623, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:17:03.690448 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.676128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:39099 (local address 127.0.0.1:49824)
0806 02:17:03.676603 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.676656 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:03.677606 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:03.678428 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:03.678433 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:03.678766 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:03.679138 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:03.679155 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.680645 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.680651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:03.681902 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:03.681919 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.682590 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.682594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:03.682753 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:03.683387 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:03.683403 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:03.685122 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:03.687458 (+ 2336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:03.687467 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:03.687475 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:03.687775 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:03.688266 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:03.688269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:03.688271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:03.688383 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:03.689154 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:03.689158 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:03.689291 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 02:17:03.689419 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 02:17:03.689651 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":59,"thread_start_us":127,"threads_started":1}
I20260806 02:17:03.690451 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.677067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49824 (local address 127.18.70.126:39099)
0806 02:17:03.677392 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:03.677406 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:03.677457 (+ 51us) server_negotiation.cc:407] Connection header received
0806 02:17:03.677934 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:03.677942 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:03.678216 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:03.678384 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:03.679291 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.680479 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.682054 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.682430 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.682605 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:03.685281 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:03.685292 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:03.685310 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:03.685344 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:03.687345 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:03.687888 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:03.687898 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:03.687900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:03.688038 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:03.688503 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:03.688508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:03.688511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:03.689037 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:03.689192 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:03.689568 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 02:17:03.689740 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":200,"thread_start_us":93,"threads_started":1}
I20260806 02:17:03.693405 10319 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:17:03.694165 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.693912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40561 (local address 127.0.0.1:47356)
0806 02:17:03.694024 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.694109 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 02:17:03.694454 10265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40561: connect: Connection refused (error 111)
I20260806 02:17:03.695284 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.695078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40561 (local address 127.0.0.1:47372)
0806 02:17:03.695172 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.695214 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 02:17:03.695523 10319 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.125:40561: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40561: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.125.keytab.
Entry for principal kudu/127.18.70.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.125.keytab.
I20260806 02:17:03.699038 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.125:40561
--webserver_interface=127.18.70.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.125.keytab
--principal=kudu/127.18.70.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:46427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:39099,127.18.70.125:40561,127.18.70.124:39837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:17:03.744850 10319 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } attempt: 1
I20260806 02:17:03.745745 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.745409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40561 (local address 127.0.0.1:47388)
0806 02:17:03.745579 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.745652 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 02:17:03.746465 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.746243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40561 (local address 127.0.0.1:47402)
0806 02:17:03.746346 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.746397 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 02:17:03.746608 10319 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.125:40561: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40561: connect: Connection refused (error 111)
W20260806 02:17:03.809355 10328 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 02:17:03.809654 10328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:17:03.809677 10328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:17:03.809720 10328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:17:03.813546 10328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:17:03.813593 10328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:17:03.813611 10328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:17:03.813627 10328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:17:03.813642 10328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:17:03.814788 10319 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } attempt: 2
I20260806 02:17:03.815613 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.815271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40561 (local address 127.0.0.1:47414)
0806 02:17:03.815424 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.815507 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 02:17:03.816289 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.816077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40561 (local address 127.0.0.1:47420)
0806 02:17:03.816174 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.816227 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 02:17:03.816411 10319 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.125:40561: Network error: Client connection negotiation failed: client connection to 127.18.70.125:40561: connect: Connection refused (error 111)
I20260806 02:17:03.819015 10328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:46427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:39099,127.18.70.125:40561,127.18.70.124:39837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.125.keytab
--principal=kudu/127.18.70.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.125:40561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.70.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:17:03.819993 10328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:17:03.820888 10328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:17:03.828951 10334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:17:03.828969 10333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:17:03.828941 10336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:17:03.829360 10328 server_base.cc:1034] running on GCE node
I20260806 02:17:03.829929 10328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:17:03.831009 10328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:17:03.832170 10328 hybrid_clock.cc:648] HybridClock initialized: now 1785982623832132 us; error 48 us; skew 500 ppm
Aug 06 02:17:03 dist-test-slave-6k22 krb5kdc[10233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982623, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:17:03.834777 10328 init.cc:377] Logged in from keytab as kudu/127.18.70.125@KRBTEST.COM (short username kudu)
I20260806 02:17:03.835656 10328 webserver.cc:460] Webserver started at http://127.18.70.125:44189/ using document root <none> and password file <none>
I20260806 02:17:03.836163 10328 fs_manager.cc:359] Metadata directory not provided
I20260806 02:17:03.836223 10328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:17:03.836386 10328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:17:03.838284 10328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/data/instance:
uuid: "50758df00b0d44469fa67f6f5146c8d9"
format_stamp: "Formatted at 2026-08-06 02:17:03 on dist-test-slave-6k22"
server_key: "b80ae0a3096763c5dda1777dd9eda9e8"
server_key_iv: "9f63b7ec491f589a1b7ae55bc9862ef4"
server_key_version: "encryptionkey@0"
I20260806 02:17:03.838819 10328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/wal/instance:
uuid: "50758df00b0d44469fa67f6f5146c8d9"
format_stamp: "Formatted at 2026-08-06 02:17:03 on dist-test-slave-6k22"
server_key: "b80ae0a3096763c5dda1777dd9eda9e8"
server_key_iv: "9f63b7ec491f589a1b7ae55bc9862ef4"
server_key_version: "encryptionkey@0"
I20260806 02:17:03.842447 10328 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 02:17:03.844703 10343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:17:03.846742 10328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 02:17:03.846937 10328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/wal
uuid: "50758df00b0d44469fa67f6f5146c8d9"
format_stamp: "Formatted at 2026-08-06 02:17:03 on dist-test-slave-6k22"
server_key: "b80ae0a3096763c5dda1777dd9eda9e8"
server_key_iv: "9f63b7ec491f589a1b7ae55bc9862ef4"
server_key_version: "encryptionkey@0"
I20260806 02:17:03.847108 10328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:17:03.887876 10328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:17:03.894501 10328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:17:03.894901 10328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:17:03.907205 10328 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.125:40561
I20260806 02:17:03.907599 10319 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } attempt: 3
I20260806 02:17:03.907735 10398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.125:40561 every 8 connection(s)
I20260806 02:17:03.910861 10328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/data/info.pb
I20260806 02:17:03.916417 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 10328
I20260806 02:17:03.916507 10400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:17:03.916618 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-1/wal/instance
I20260806 02:17:03.917102 18713 external_mini_cluster.cc:1428] Setting key 9220ca89234d49eff78b5d57f3c783c2
Aug 06 02:17:03 dist-test-slave-6k22 krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982623, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for kudu/127.18.70.125@KRBTEST.COM
I20260806 02:17:03.925859 10400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:17:03.929893 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.908466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40561 (local address 127.0.0.1:47424)
0806 02:17:03.908780 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.908837 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:03.909019 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:03.913345 (+ 4326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:03.913350 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:03.913700 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:03.914030 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:03.914050 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.916198 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.916204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:03.917524 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:03.917537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.919671 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.919679 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:03.919878 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:03.920848 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:03.920880 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:03.923130 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:03.926465 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:03.926470 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:03.926472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:03.926917 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:03.927625 (+ 708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:03.927629 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:03.927630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:03.927776 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:03.928947 (+ 1171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:03.928951 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:03.929140 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 02:17:03.929332 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 02:17:03.929567 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 02:17:03.934190 10399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.909740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47424 (local address 127.18.70.125:40561)
0806 02:17:03.910508 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:03.910538 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:03.910606 (+ 68us) server_negotiation.cc:407] Connection header received
0806 02:17:03.912197 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:03.912236 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:03.912879 (+ 643us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:03.913357 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:03.914241 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.915906 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.917766 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.919410 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.919790 (+ 380us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:03.923470 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:03.923485 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:03.923507 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:03.923543 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:03.926199 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:03.927133 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:03.927151 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:03.927162 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:03.927362 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:03.927956 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:03.927962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:03.927966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:03.928746 (+ 780us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:03.929000 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:03.931536 (+ 2536us) server_negotiation.cc:299] Negotiation successful
0806 02:17:03.932145 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":273,"threads_started":1}
WARNING: no policy specified for kudu/127.18.70.124@KRBTEST.COM; defaulting to no policy
I20260806 02:17:03.937551 10319 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:17:03.938853 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.938392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:60878)
0806 02:17:03.938617 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.938708 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.70.124@KRBTEST.COM" created.
I20260806 02:17:03.940388 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.939822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:60882)
0806 02:17:03.940115 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.940253 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
Aug 06 02:17:03 dist-test-slave-6k22 krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982623, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.125@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
W20260806 02:17:03.943198 10319 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:39837: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
I20260806 02:17:03.949852 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.929921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:39099 (local address 127.0.0.1:49838)
0806 02:17:03.931856 (+ 1935us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.931942 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:03.932166 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:03.932681 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:03.932688 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:03.933597 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:03.933871 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:03.933899 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.935895 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.935899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:03.937129 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:03.937142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.938167 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.938177 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:03.938436 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:03.939728 (+ 1292us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:03.939740 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:03.943190 (+ 3450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:03.946174 (+ 2984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:03.946188 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:03.946196 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:03.946805 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:03.947290 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:03.947298 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:03.947303 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:03.947500 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:03.948212 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:03.948233 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:03.948521 (+ 288us) client_negotiation.cc:769] Sending connection context
0806 02:17:03.948848 (+ 327us) client_negotiation.cc:240] Negotiation successful
0806 02:17:03.949283 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1639,"thread_start_us":235,"threads_started":1}
I20260806 02:17:03.950254 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.931293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49838 (local address 127.18.70.126:39099)
0806 02:17:03.931633 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:03.931640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:03.932086 (+ 446us) server_negotiation.cc:407] Connection header received
0806 02:17:03.932342 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:03.932348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:03.932476 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:03.932632 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:03.934100 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.935540 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.937482 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.937950 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.938168 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:03.943532 (+ 5364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:03.943543 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:03.943545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:03.943628 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:03.946000 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:03.946996 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:03.947007 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:03.947008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:03.947119 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:03.947729 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:03.947731 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:03.947733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:03.948016 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:03.948140 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:03.949767 (+ 1627us) server_negotiation.cc:299] Negotiation successful
0806 02:17:03.949920 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 02:17:03.952174 10400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 02:17:03 dist-test-slave-6k22 krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982623, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.125@KRBTEST.COM for kudu/127.18.70.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.70.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.124.keytab.
Entry for principal kudu/127.18.70.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.124.keytab.
I20260806 02:17:03.966077 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.952822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40561 (local address 127.0.0.1:47430)
0806 02:17:03.953199 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.953234 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:03.953345 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:03.953784 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:03.953787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:03.954005 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:03.954221 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:03.954241 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.956076 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.956079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:03.956922 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:03.956932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.957790 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.957801 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:03.957935 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:03.958564 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:03.958608 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:03.960870 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:03.963449 (+ 2579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:03.963452 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:03.963454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:03.963879 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:03.964311 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:03.964313 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:03.964314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:03.964438 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:03.965053 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:03.965055 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:03.965224 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 02:17:03.965430 (+ 206us) client_negotiation.cc:240] Negotiation successful
0806 02:17:03.965640 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260806 02:17:03.966104 10399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.953013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47430 (local address 127.18.70.125:40561)
0806 02:17:03.953194 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:03.953201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:03.953339 (+ 138us) server_negotiation.cc:407] Connection header received
0806 02:17:03.953472 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:03.953477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:03.953583 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:03.953772 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:03.954435 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.955923 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.957091 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:03.957624 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:03.957742 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:03.961079 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:03.961087 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:03.961093 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:03.961146 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:03.963285 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:03.964053 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:03.964056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:03.964057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:03.964172 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:03.964547 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:03.964550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:03.964551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:03.964906 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:03.965051 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:03.965526 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 02:17:03.965744 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":53}
I20260806 02:17:03.967522 10400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:17:03.967541 18713 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
/tmp/dist-test-taskVlzDnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.124:39837
--webserver_interface=127.18.70.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.124.keytab
--principal=kudu/127.18.70.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.70.84:46427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.70.126:39099,127.18.70.125:40561,127.18.70.124:39837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 02:17:03.968888 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.968473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:60890)
0806 02:17:03.968655 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.968809 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 02:17:03.969372 10345 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
I20260806 02:17:03.970956 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.970616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:60896)
0806 02:17:03.970805 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.970875 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 02:17:03.971230 10400 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:39837: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
I20260806 02:17:03.974457 10319 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } attempt: 1
I20260806 02:17:03.975732 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.975110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:60906)
0806 02:17:03.975369 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.975486 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 02:17:03.977056 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:03.976578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:60922)
0806 02:17:03.976772 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:03.976869 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 02:17:03.977303 10319 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:39837: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
I20260806 02:17:04.020578 10400 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } attempt: 1
I20260806 02:17:04.022006 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.021346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:60938)
0806 02:17:04.021635 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.021808 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 02:17:04.023403 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.022901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:60944)
0806 02:17:04.023085 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.023240 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 02:17:04.023680 10400 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:39837: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
I20260806 02:17:04.052524 10319 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } attempt: 2
I20260806 02:17:04.053917 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.053397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:60960)
0806 02:17:04.053697 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.053776 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260806 02:17:04.055059 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.054696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:60976)
0806 02:17:04.054881 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.054969 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 02:17:04.055271 10319 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:39837: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
I20260806 02:17:04.091980 10400 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } attempt: 2
I20260806 02:17:04.093379 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.092944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:60990)
0806 02:17:04.093179 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.093279 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 02:17:04.094518 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.094166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:60994)
0806 02:17:04.094336 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.094421 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 02:17:04.094682 10400 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:39837: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
I20260806 02:17:04.154469 10319 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } attempt: 3
I20260806 02:17:04.155755 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.155235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:32772)
0806 02:17:04.155468 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.155637 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 02:17:04.156931 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.156660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:32778)
0806 02:17:04.156780 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.156849 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 02:17:04.157258 10319 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:39837: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
I20260806 02:17:04.191627 10400 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } attempt: 3
I20260806 02:17:04.193001 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.192539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:32792)
0806 02:17:04.192743 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.192855 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 02:17:04.194351 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.193896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:32800)
0806 02:17:04.194153 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.194240 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260806 02:17:04.194619 10400 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.70.124:39837: Network error: Client connection negotiation failed: client connection to 127.18.70.124:39837: connect: Connection refused (error 111)
W20260806 02:17:04.210529 10407 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 02:17:04.211048 10407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 02:17:04.211098 10407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 02:17:04.211179 10407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 02:17:04.217772 10407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 02:17:04.217872 10407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 02:17:04.217916 10407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 02:17:04.218037 10407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 02:17:04.218128 10407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 02:17:04.227232 10407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.70.84:46427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.70.126:39099,127.18.70.125:40561,127.18.70.124:39837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/krb5kdc/kudu_127.18.70.124.keytab
--principal=kudu/127.18.70.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.70.124:39837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.70.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 01:37:46 UTC on ccycloud.quasar-negaek.root.comops.site
build id 4569534
I20260806 02:17:04.229501 10407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 02:17:04.231638 10407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 02:17:04.247375 10412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:17:04.247391 10415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 02:17:04.247483 10413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 02:17:04.247967 10407 server_base.cc:1034] running on GCE node
I20260806 02:17:04.248994 10407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 02:17:04.250934 10407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 02:17:04.252220 10407 hybrid_clock.cc:648] HybridClock initialized: now 1785982624252152 us; error 59 us; skew 500 ppm
Aug 06 02:17:04 dist-test-slave-6k22 krb5kdc[10233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982624, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 02:17:04.257308 10407 init.cc:377] Logged in from keytab as kudu/127.18.70.124@KRBTEST.COM (short username kudu)
I20260806 02:17:04.258960 10407 webserver.cc:460] Webserver started at http://127.18.70.124:45181/ using document root <none> and password file <none>
I20260806 02:17:04.260088 10407 fs_manager.cc:359] Metadata directory not provided
I20260806 02:17:04.260242 10407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 02:17:04.260602 10407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 02:17:04.264223 10407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/data/instance:
uuid: "2d62bb161aa041659a0f57142cc91ef5"
format_stamp: "Formatted at 2026-08-06 02:17:04 on dist-test-slave-6k22"
server_key: "53e2bf01477f287a0f3eb8a3d9672a59"
server_key_iv: "e3f8fff30a968ee0fc193c70570422ec"
server_key_version: "encryptionkey@0"
I20260806 02:17:04.265208 10407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/wal/instance:
uuid: "2d62bb161aa041659a0f57142cc91ef5"
format_stamp: "Formatted at 2026-08-06 02:17:04 on dist-test-slave-6k22"
server_key: "53e2bf01477f287a0f3eb8a3d9672a59"
server_key_iv: "e3f8fff30a968ee0fc193c70570422ec"
server_key_version: "encryptionkey@0"
I20260806 02:17:04.272557 10407 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 02:17:04.276932 10422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 02:17:04.279247 10407 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 02:17:04.279675 10407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/data,/tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/wal
uuid: "2d62bb161aa041659a0f57142cc91ef5"
format_stamp: "Formatted at 2026-08-06 02:17:04 on dist-test-slave-6k22"
server_key: "53e2bf01477f287a0f3eb8a3d9672a59"
server_key_iv: "e3f8fff30a968ee0fc193c70570422ec"
server_key_version: "encryptionkey@0"
I20260806 02:17:04.279901 10407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 02:17:04.297075 10407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 02:17:04.302486 10407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 02:17:04.302796 10407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 02:17:04.315271 10407 rpc_server.cc:307] RPC server started. Bound to: 127.18.70.124:39837
I20260806 02:17:04.315747 10475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.70.124:39837 every 8 connection(s)
I20260806 02:17:04.317054 10407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/data/info.pb
I20260806 02:17:04.321483 10319 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } attempt: 4
I20260806 02:17:04.322167 10476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 02:17:04.325465 18713 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu as pid 10407
I20260806 02:17:04.325695 18713 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskVlzDnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785982339772911-18713-0/minicluster-data/master-2/wal/instance
I20260806 02:17:04.326431 18713 external_mini_cluster.cc:1428] Setting key 79c8952b6d55025025149289f34d0073
I20260806 02:17:04.332073 10476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 02:17:04 dist-test-slave-6k22 krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982623, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.126@KRBTEST.COM for kudu/127.18.70.124@KRBTEST.COM
I20260806 02:17:04.337852 10400 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } attempt: 4
Aug 06 02:17:04 dist-test-slave-6k22 krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.124@KRBTEST.COM
Aug 06 02:17:04 dist-test-slave-6k22 krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982623, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.125@KRBTEST.COM for kudu/127.18.70.124@KRBTEST.COM
I20260806 02:17:04.348590 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.322447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:32808)
0806 02:17:04.322677 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.322708 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:04.322859 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:04.326903 (+ 4044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:04.326919 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:04.327362 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:04.327618 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:04.327629 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.331069 (+ 3440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.331077 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:04.332412 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:04.332435 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.334317 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.334323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:04.334486 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.335431 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:04.335465 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:04.338452 (+ 2987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:04.343802 (+ 5350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:04.343820 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:04.343824 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:04.344388 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:04.345568 (+ 1180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:04.345585 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:04.345588 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:04.345805 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:04.347398 (+ 1593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:04.347402 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:04.347669 (+ 267us) client_negotiation.cc:769] Sending connection context
0806 02:17:04.347934 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 02:17:04.348180 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 02:17:04.351835 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.323520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.18.70.124:39837)
0806 02:17:04.324294 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.324331 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.324425 (+ 94us) server_negotiation.cc:407] Connection header received
0806 02:17:04.325515 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.325545 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.326351 (+ 806us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.327038 (+ 687us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.327949 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.330830 (+ 2881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.332640 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.334078 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.334439 (+ 361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.338870 (+ 4431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:04.338897 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:04.338935 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:04.339010 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:04.343520 (+ 4510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.344990 (+ 1470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.345018 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.345025 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.345370 (+ 345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.346061 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.346065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.346066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.347173 (+ 1107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:04.347454 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:04.349153 (+ 1699us) server_negotiation.cc:299] Negotiation successful
0806 02:17:04.349714 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":483,"thread_start_us":269,"threads_started":1}
I20260806 02:17:04.352260 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.328032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32824 (local address 127.18.70.124:39837)
0806 02:17:04.328566 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.328588 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.328853 (+ 265us) server_negotiation.cc:407] Connection header received
0806 02:17:04.329118 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.329131 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.329199 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.329365 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.331106 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.331923 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.333771 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.334367 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.334564 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.340204 (+ 5640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:04.340219 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:04.340226 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:04.340300 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:04.344330 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.345370 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.345374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.345376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.345531 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.346269 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.346272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.346274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.347740 (+ 1466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:04.348146 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:04.348988 (+ 842us) server_negotiation.cc:299] Negotiation successful
0806 02:17:04.351843 (+ 2855us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2939,"server-negotiator.queue_time_us":393,"thread_start_us":194,"threads_started":1}
Aug 06 02:17:04 dist-test-slave-6k22 krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982624, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.124@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:17:04.357643 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.338702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:32826)
0806 02:17:04.339299 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.339351 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:04.339497 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:04.340338 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:04.340344 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:04.340699 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:04.340937 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:04.340954 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.342627 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.342631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:04.344032 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:04.344054 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.345112 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.345118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:04.345317 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.346226 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:04.346269 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:04.348955 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:04.354604 (+ 5649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:04.354610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:04.354615 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:04.355125 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:04.355803 (+ 678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:04.355810 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:04.355827 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:04.356027 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:04.356861 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:04.356865 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:04.357078 (+ 213us) client_negotiation.cc:769] Sending connection context
0806 02:17:04.357267 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 02:17:04.357483 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448}
I20260806 02:17:04.357921 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.338806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32826 (local address 127.18.70.124:39837)
0806 02:17:04.339837 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.339849 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.339884 (+ 35us) server_negotiation.cc:407] Connection header received
0806 02:17:04.339946 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.339958 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.340044 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.340872 (+ 828us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.341127 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.342399 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.344286 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.344867 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.345719 (+ 852us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.349242 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:04.349247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:04.349250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:04.349295 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:04.354364 (+ 5069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.355359 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.355362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.355363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.355530 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.356234 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.356247 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.356249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.356666 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:04.356881 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:04.357415 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 02:17:04.357607 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":892,"thread_start_us":171,"threads_started":1}
I20260806 02:17:04.360733 10319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251: Bootstrap starting.
I20260806 02:17:04.361704 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.335811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49840 (local address 127.18.70.126:39099)
0806 02:17:04.336017 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.336032 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.336789 (+ 757us) server_negotiation.cc:407] Connection header received
0806 02:17:04.337189 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.337196 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.337331 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.337453 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.341900 (+ 4447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.343371 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.346329 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.346929 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.347104 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.353841 (+ 6737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:04.353845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:04.353847 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:04.353896 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:04.356846 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.358838 (+ 1992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.358842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.358844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.358963 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.359458 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.359461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.359463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.360030 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:04.360228 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:04.361173 (+ 945us) server_negotiation.cc:299] Negotiation successful
0806 02:17:04.361421 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 02:17:04.363080 10482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.335725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.126:39099 (local address 127.0.0.1:49840)
0806 02:17:04.336514 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.336645 (+ 131us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:04.336998 (+ 353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:04.337846 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:04.337866 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:04.340548 (+ 2682us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:04.341717 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:04.341740 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.343615 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.343629 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:04.345488 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:04.345521 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.348635 (+ 3114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.348654 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:04.349265 (+ 611us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.350644 (+ 1379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:04.350672 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:04.353598 (+ 2926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:04.358128 (+ 4530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:04.358150 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:04.358170 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:04.358632 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:04.359117 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:04.359122 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:04.359124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:04.359270 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:04.360371 (+ 1101us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:04.360379 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:04.360638 (+ 259us) client_negotiation.cc:769] Sending connection context
0806 02:17:04.362319 (+ 1681us) client_negotiation.cc:240] Negotiation successful
0806 02:17:04.362711 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":238,"threads_started":1}
I20260806 02:17:04.365440 10476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:17:04.365543 10400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9: Bootstrap starting.
I20260806 02:17:04.367038 10319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251: Neither blocks nor log segments found. Creating new log.
I20260806 02:17:04.369004 10319 log.cc:824] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251: Log is configured to *not* fsync() on all Append() calls
I20260806 02:17:04.371958 10400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9: Neither blocks nor log segments found. Creating new log.
I20260806 02:17:04.373790 10400 log.cc:824] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9: Log is configured to *not* fsync() on all Append() calls
Aug 06 02:17:04 dist-test-slave-6k22 krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.126@KRBTEST.COM
I20260806 02:17:04.378885 10400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9: No bootstrap required, opened a new log
I20260806 02:17:04.379652 10319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251: No bootstrap required, opened a new log
Aug 06 02:17:04 dist-test-slave-6k22 krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982624, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.124@KRBTEST.COM for kudu/127.18.70.125@KRBTEST.COM
I20260806 02:17:04.384702 10484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.360491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49848 (local address 127.18.70.126:39099)
0806 02:17:04.361526 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.361540 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.361564 (+ 24us) server_negotiation.cc:407] Connection header received
0806 02:17:04.361651 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.361654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.361741 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.363413 (+ 1672us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.363665 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.364587 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.368641 (+ 4054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.369277 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.369536 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.375970 (+ 6434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:04.375986 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:04.375988 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:04.376025 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:04.380092 (+ 4067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.381206 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.381212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.381214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.381424 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.382240 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.382243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.382245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.382695 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:04.382975 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:04.383997 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0806 02:17:04.384283 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":920,"thread_start_us":147,"threads_started":1}
I20260806 02:17:04.389698 10400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } }
I20260806 02:17:04.390396 10400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50758df00b0d44469fa67f6f5146c8d9, State: Initialized, Role: FOLLOWER
I20260806 02:17:04.390172 10319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } }
I20260806 02:17:04.390621 10319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71b9558f5262453aaf5b11023fdc0251, State: Initialized, Role: FOLLOWER
I20260806 02:17:04.390897 10482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.366727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.125:40561 (local address 127.0.0.1:47442)
0806 02:17:04.370064 (+ 3337us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.370100 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:04.370965 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:04.371801 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:04.371806 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:04.372244 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:04.372533 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:04.372554 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.374400 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.374406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:04.375419 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:04.375436 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.376867 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.376882 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:04.377105 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.378587 (+ 1482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:04.378618 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:04.381972 (+ 3354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:04.387008 (+ 5036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:04.387018 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:04.387020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:04.387687 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:04.388390 (+ 703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:04.388413 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:04.388426 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:04.388655 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:04.389837 (+ 1182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:04.389841 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:04.390065 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 02:17:04.390286 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 02:17:04.390562 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3176}
I20260806 02:17:04.391040 10399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.366846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47442 (local address 127.18.70.125:40561)
0806 02:17:04.367041 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.367057 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.370836 (+ 3779us) server_negotiation.cc:407] Connection header received
0806 02:17:04.371260 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.371263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.371353 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.371472 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.372811 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.374040 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.375686 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.376272 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.376513 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.382148 (+ 5635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:04.382152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:04.382155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:04.382178 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:04.386130 (+ 3952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.387979 (+ 1849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.387984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.387986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.388153 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.388931 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.388944 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.388946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.389456 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:04.389721 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:04.390471 (+ 750us) server_negotiation.cc:299] Negotiation successful
0806 02:17:04.390723 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 02:17:04.392112 10400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } }
I20260806 02:17:04.392114 10319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } }
I20260806 02:17:04.392448 10476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } has no permanent_uuid. Determining permanent_uuid...
I20260806 02:17:04.396166 10319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:17:04.396179 10400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 02:17:04.397709 10349 tablet.cc:2369] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 02:17:04 dist-test-slave-6k22 krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982624, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.70.124@KRBTEST.COM for kudu/127.18.70.124@KRBTEST.COM
I20260806 02:17:04.400928 10488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } } }
I20260806 02:17:04.405961 10488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 02:17:04 dist-test-slave-6k22 krb5kdc[10233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785982623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.70.125@KRBTEST.COM
W20260806 02:17:04.410763 10269 tablet.cc:2369] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:17:04.417412 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.393978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32842 (local address 127.18.70.124:39837)
0806 02:17:04.394230 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.394264 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.394432 (+ 168us) server_negotiation.cc:407] Connection header received
0806 02:17:04.394891 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.394894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.395015 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.395149 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.396372 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.397710 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.399800 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.400524 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.400684 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.408081 (+ 7397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:04.408095 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:04.408098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:04.408155 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:04.413209 (+ 5054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.414734 (+ 1525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.414738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.414739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.414947 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.415634 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.415639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.415641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.416118 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:04.416326 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:04.416743 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 02:17:04.417066 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 02:17:04.418489 10482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.393430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.70.124:39837 (local address 127.0.0.1:32842)
0806 02:17:04.394220 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0806 02:17:04.394293 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 02:17:04.394585 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 02:17:04.395455 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 02:17:04.395459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 02:17:04.395913 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0806 02:17:04.396113 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:04.396120 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.398034 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.398037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:04.399502 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 02:17:04.399526 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.401012 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.401030 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 02:17:04.401263 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.402427 (+ 1164us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 02:17:04.402480 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0806 02:17:04.407715 (+ 5235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 02:17:04.413683 (+ 5968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:04.413697 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:04.413701 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:04.414487 (+ 786us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:04.415251 (+ 764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 02:17:04.415260 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 02:17:04.415262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 02:17:04.415390 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 02:17:04.416433 (+ 1043us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 02:17:04.416439 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 02:17:04.416534 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 02:17:04.417869 (+ 1335us) client_negotiation.cc:240] Negotiation successful
0806 02:17:04.418117 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678}
W20260806 02:17:04.418756 10513 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:17:04.418931 10513 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:17:04.420486 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.388262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47454 (local address 127.18.70.125:40561)
0806 02:17:04.389114 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.389140 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.391523 (+ 2383us) server_negotiation.cc:407] Connection header received
0806 02:17:04.391988 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.392011 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.392112 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.392331 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.394758 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.396266 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.398435 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.399204 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.399455 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.408591 (+ 9136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:04.408595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:04.408598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:04.408654 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:04.414993 (+ 6339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.416146 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.416150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.416152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.416347 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.416730 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.416745 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.416747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.417302 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:04.419136 (+ 1834us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:04.419306 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 02:17:04.419539 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":317,"threads_started":1}
I20260806 02:17:04.425390 10476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5: Bootstrap starting.
I20260806 02:17:04.427174 18713 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 02:17:04.429955 10290 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "71b9558f5262453aaf5b11023fdc0251"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49848
I20260806 02:17:04.430441 10290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 02:17:04.430672 10290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:17:04.432279 10476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5: Neither blocks nor log segments found. Creating new log.
W20260806 02:17:04.432890 10515 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:17:04.433105 10515 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:17:04.421737 10248 master_runner.cc:186] Error getting master registration for 127.18.70.124:39837: OK, instance_id { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" instance_seqno: 1785982624311950 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 02:17:04.434140 10489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } } }
I20260806 02:17:04.433921 10290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } }
I20260806 02:17:04.434512 10476 log.cc:824] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5: Log is configured to *not* fsync() on all Append() calls
I20260806 02:17:04.438670 10290 leader_election.cc:288] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [CANDIDATE]: Term 1 election: Requested vote from peers 50758df00b0d44469fa67f6f5146c8d9 (127.18.70.125:40561), 2d62bb161aa041659a0f57142cc91ef5 (127.18.70.124:39837)
I20260806 02:17:04.439456 10248 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:17:04.439587 10476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5: No bootstrap required, opened a new log
I20260806 02:17:04.437731 10370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71b9558f5262453aaf5b11023fdc0251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "50758df00b0d44469fa67f6f5146c8d9"
I20260806 02:17:04.440066 10370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:17:04.443339 10370 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71b9558f5262453aaf5b11023fdc0251 in term 1.
I20260806 02:17:04.436812 10328 master_runner.cc:186] Error getting master registration for 127.18.70.124:39837: OK, instance_id { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" instance_seqno: 1785982624311950 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 02:17:04.444875 10328 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 02:17:04.446105 10489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:17:04.445783 10265 leader_election.cc:302] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50758df00b0d44469fa67f6f5146c8d9, 71b9558f5262453aaf5b11023fdc0251; no voters:
I20260806 02:17:04.446800 10488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 02:17:04.447360 10488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [term 1 LEADER]: Becoming Leader. State: Replica: 71b9558f5262453aaf5b11023fdc0251, State: Running, Role: LEADER
I20260806 02:17:04.448227 10488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } }
I20260806 02:17:04.452793 10488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71b9558f5262453aaf5b11023fdc0251. Latest consensus state: current_term: 1 leader_uuid: "71b9558f5262453aaf5b11023fdc0251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } } }
I20260806 02:17:04.453104 10488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [sys.catalog]: This master's current role is: LEADER
I20260806 02:17:04.454252 10476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } }
I20260806 02:17:04.454916 10476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d62bb161aa041659a0f57142cc91ef5, State: Initialized, Role: FOLLOWER
I20260806 02:17:04.458822 10476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } }
I20260806 02:17:04.462304 10476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 02:17:04.462432 10449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "71b9558f5262453aaf5b11023fdc0251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2d62bb161aa041659a0f57142cc91ef5"
I20260806 02:17:04.462918 10449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 02:17:04.466182 10449 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71b9558f5262453aaf5b11023fdc0251 in term 1.
I20260806 02:17:04.470835 10530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 02:17:04.473665 10530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 02:17:04.475677 10531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } } }
I20260806 02:17:04.475950 10531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:17:04.493301 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.451618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47462 (local address 127.18.70.125:40561)
0806 02:17:04.451873 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.451891 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.456186 (+ 4295us) server_negotiation.cc:407] Connection header received
0806 02:17:04.456752 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.456756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.456898 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.457008 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.458596 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.460223 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.464552 (+ 4329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.465358 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.465587 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.476362 (+ 10775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:04.476381 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:04.476396 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:04.476466 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:04.480489 (+ 4023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.487124 (+ 6635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.487130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.487132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.487420 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.489264 (+ 1844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.489277 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.489279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.489772 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:04.490181 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:04.492584 (+ 2403us) server_negotiation.cc:299] Negotiation successful
0806 02:17:04.492824 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 02:17:04.498389 10484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.449768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49856 (local address 127.18.70.126:39099)
0806 02:17:04.450082 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.450106 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.453967 (+ 3861us) server_negotiation.cc:407] Connection header received
0806 02:17:04.454330 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.454344 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.454465 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.454618 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.457298 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.459804 (+ 2506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.467280 (+ 7476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.468124 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.468336 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.481850 (+ 13514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:04.481865 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:04.481871 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:04.481918 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:04.485789 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.488351 (+ 2562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.488366 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.488368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.488547 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.490894 (+ 2347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.490898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.490899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.491387 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:04.491635 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:04.496853 (+ 5218us) server_negotiation.cc:299] Negotiation successful
0806 02:17:04.497216 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 02:17:04.510826 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:04.453087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32854 (local address 127.18.70.124:39837)
0806 02:17:04.453375 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.453391 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.464044 (+ 10653us) server_negotiation.cc:407] Connection header received
0806 02:17:04.464531 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.464535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.464616 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.464751 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.479261 (+ 14510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.480746 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.495219 (+ 14473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:04.495854 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:04.496067 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:04.502530 (+ 6463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 02:17:04.502536 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 02:17:04.502538 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 02:17:04.502621 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0806 02:17:04.506440 (+ 3819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.507494 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.507509 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.507510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.507756 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 02:17:04.508516 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 02:17:04.508520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 02:17:04.508531 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 02:17:04.509017 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 02:17:04.509305 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:04.510142 (+ 837us) server_negotiation.cc:299] Negotiation successful
0806 02:17:04.510424 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
W20260806 02:17:04.520275 10543 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 02:17:04.520429 10543 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 02:17:04.525537 10449 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5 [term 1 FOLLOWER]: Refusing update from remote peer 71b9558f5262453aaf5b11023fdc0251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:17:04.527212 10370 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9 [term 1 FOLLOWER]: Refusing update from remote peer 71b9558f5262453aaf5b11023fdc0251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 02:17:04.529026 10488 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:17:04.530015 10488 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 02:17:04.546945 10531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71b9558f5262453aaf5b11023fdc0251. Latest consensus state: current_term: 1 leader_uuid: "71b9558f5262453aaf5b11023fdc0251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } } }
I20260806 02:17:04.547222 10531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:17:04.552098 10489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71b9558f5262453aaf5b11023fdc0251. Latest consensus state: current_term: 1 leader_uuid: "71b9558f5262453aaf5b11023fdc0251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } } }
I20260806 02:17:04.552451 10489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 02:17:04.554984 10428 tablet.cc:2369] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 02:17:04.571929 10489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "71b9558f5262453aaf5b11023fdc0251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } } }
I20260806 02:17:04.572238 10489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:17:04.574524 10524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "71b9558f5262453aaf5b11023fdc0251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } } }
I20260806 02:17:04.574862 10524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [sys.catalog]: This master's current role is: LEADER
I20260806 02:17:04.577430 10488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "71b9558f5262453aaf5b11023fdc0251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } } }
I20260806 02:17:04.577674 10488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251 [sys.catalog]: This master's current role is: LEADER
I20260806 02:17:04.580634 10531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "71b9558f5262453aaf5b11023fdc0251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71b9558f5262453aaf5b11023fdc0251" member_type: VOTER last_known_addr { host: "127.18.70.126" port: 39099 } } peers { permanent_uuid: "50758df00b0d44469fa67f6f5146c8d9" member_type: VOTER last_known_addr { host: "127.18.70.125" port: 40561 } } peers { permanent_uuid: "2d62bb161aa041659a0f57142cc91ef5" member_type: VOTER last_known_addr { host: "127.18.70.124" port: 39837 } } }
I20260806 02:17:04.580924 10531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 02:17:04.582966 10546 mvcc.cc:205] Tried to move back new op lower bound from 7315384830040555520 to 7315384829747527680. Current Snapshot: MvccSnapshot[applied={T|T < 7315384830040555520}]
I20260806 02:17:04.585415 10545 mvcc.cc:205] Tried to move back new op lower bound from 7315384830040555520 to 7315384829747527680. Current Snapshot: MvccSnapshot[applied={T|T < 7315384830040555520}]
I20260806 02:17:04.588918 10530 catalog_manager.cc:1347] Generated new cluster ID: 34cd50981ab148cc81b4696b706d4103
I20260806 02:17:04.588999 10530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 02:17:04.654789 10530 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 02:17:04.657639 10530 catalog_manager.cc:1504] Loading token signing keys...
I20260806 02:17:04.708081 10530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71b9558f5262453aaf5b11023fdc0251: Generated new TSK 0
I20260806 02:17:04.710194 10530 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 02:17:04.804119 10484 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:04.797215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49864 (local address 127.18.70.126:39099)
0806 02:17:04.797481 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.797505 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.801380 (+ 3875us) server_negotiation.cc:407] Connection header received
0806 02:17:04.801888 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.801892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.802020 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.802146 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.803815 (+ 1669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49864: BlockingRecv error: recv got EOF from 127.0.0.1:49864 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 02:17:04.810005 10487 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:04.798812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47466 (local address 127.18.70.125:40561)
0806 02:17:04.799069 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.799093 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.804927 (+ 5834us) server_negotiation.cc:407] Connection header received
0806 02:17:04.807281 (+ 2354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.807285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.807403 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.807634 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.809718 (+ 2084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47466: BlockingRecv error: recv got EOF from 127.0.0.1:47466 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 02:17:04.811978 10483 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:04.800212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32856 (local address 127.18.70.124:39837)
0806 02:17:04.800478 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.800485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.805401 (+ 4916us) server_negotiation.cc:407] Connection header received
0806 02:17:04.808100 (+ 2699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.808104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.808243 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.808373 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.811730 (+ 3357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32856: BlockingRecv error: recv got EOF from 127.0.0.1:32856 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 02:17:04.823602 10487 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:04.820059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47476 (local address 127.18.70.125:40561)
0806 02:17:04.820297 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.820302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.821340 (+ 1038us) server_negotiation.cc:407] Connection header received
0806 02:17:04.821820 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.821835 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.821994 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.822098 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.823366 (+ 1268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47476: BlockingRecv error: recv got EOF from 127.0.0.1:47476 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
W20260806 02:17:04.834208 10487 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:04.831119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47486 (local address 127.18.70.125:40561)
0806 02:17:04.831233 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.831238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.832242 (+ 1004us) server_negotiation.cc:407] Connection header received
0806 02:17:04.832606 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.832619 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.832725 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.832905 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.833982 (+ 1077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47486: BlockingRecv error: recv got EOF from 127.0.0.1:47486 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 02:17:04.844790 10487 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:04.840939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47498 (local address 127.18.70.125:40561)
0806 02:17:04.841243 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.841267 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.842623 (+ 1356us) server_negotiation.cc:407] Connection header received
0806 02:17:04.843153 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.843158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.843312 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.843431 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.844596 (+ 1165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47498: BlockingRecv error: recv got EOF from 127.0.0.1:47498 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
W20260806 02:17:04.857556 10487 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:04.853645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47512 (local address 127.18.70.125:40561)
0806 02:17:04.853949 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.853965 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.855122 (+ 1157us) server_negotiation.cc:407] Connection header received
0806 02:17:04.855499 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.855513 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.855670 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.855776 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.857288 (+ 1512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47512: BlockingRecv error: recv got EOF from 127.0.0.1:47512 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
W20260806 02:17:04.878082 10487 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:04.872999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47528 (local address 127.18.70.125:40561)
0806 02:17:04.873362 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.873380 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.874832 (+ 1452us) server_negotiation.cc:407] Connection header received
0806 02:17:04.875470 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.875487 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.875700 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.875879 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.877694 (+ 1815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47528: BlockingRecv error: recv got EOF from 127.0.0.1:47528 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
W20260806 02:17:04.934948 10487 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:04.916173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47542 (local address 127.18.70.125:40561)
0806 02:17:04.916487 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.916504 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.932239 (+ 15735us) server_negotiation.cc:407] Connection header received
0806 02:17:04.932857 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.932881 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.933000 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.933160 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.934710 (+ 1550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47542: BlockingRecv error: recv got EOF from 127.0.0.1:47542 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
W20260806 02:17:04.977438 10487 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:04.974529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47558 (local address 127.18.70.125:40561)
0806 02:17:04.974756 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:04.974768 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:04.974951 (+ 183us) server_negotiation.cc:407] Connection header received
0806 02:17:04.975305 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:04.975309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:04.975425 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:04.975639 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:04.977114 (+ 1475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47558: BlockingRecv error: recv got EOF from 127.0.0.1:47558 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 02:17:05.050307 10487 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:05.047337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47560 (local address 127.18.70.125:40561)
0806 02:17:05.047678 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:05.047692 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:05.048188 (+ 496us) server_negotiation.cc:407] Connection header received
0806 02:17:05.048601 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:05.048604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:05.048731 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:05.048910 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:05.050028 (+ 1118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47560: BlockingRecv error: recv got EOF from 127.0.0.1:47560 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260806 02:17:05.181846 10487 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:05.180767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47564 (local address 127.18.70.125:40561)
0806 02:17:05.180897 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:05.180901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:05.181032 (+ 131us) server_negotiation.cc:407] Connection header received
0806 02:17:05.181169 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:05.181172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:05.181222 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:05.181299 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:05.181708 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47564: BlockingRecv error: recv got EOF from 127.0.0.1:47564 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 02:17:05.434559 10515 catalog_manager.cc:1259] Loaded cluster ID: 34cd50981ab148cc81b4696b706d4103
I20260806 02:17:05.434635 10515 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9: loading cluster ID for follower catalog manager: success
I20260806 02:17:05.436401 10515 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9: acquiring CA information for follower catalog manager: success
I20260806 02:17:05.437533 10515 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 50758df00b0d44469fa67f6f5146c8d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:17:05.443868 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:05.439952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47570 (local address 127.18.70.125:40561)
0806 02:17:05.440202 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:05.440207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:05.440333 (+ 126us) server_negotiation.cc:407] Connection header received
0806 02:17:05.440506 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:05.440510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:05.440609 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:05.440736 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:17:05.441002 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:05.441498 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:05.442391 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:05.442611 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:05.442727 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:05.443060 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:17:05.443450 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:17:05.443600 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:05.443676 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 02:17:05.443736 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
W20260806 02:17:05.447791 10617 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 02:17:05.446649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.18.70.124:39837)
0806 02:17:05.446945 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:05.446949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:05.446965 (+ 16us) server_negotiation.cc:407] Connection header received
0806 02:17:05.447057 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:05.447059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:05.447136 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:05.447218 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 02:17:05.447658 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32870: BlockingRecv error: recv got EOF from 127.0.0.1:32870 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":99,"threads_started":1}
I20260806 02:17:05.521834 10543 catalog_manager.cc:1259] Loaded cluster ID: 34cd50981ab148cc81b4696b706d4103
I20260806 02:17:05.521916 10543 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5: loading cluster ID for follower catalog manager: success
I20260806 02:17:05.523733 10543 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5: acquiring CA information for follower catalog manager: success
I20260806 02:17:05.524875 10543 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2d62bb161aa041659a0f57142cc91ef5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 02:17:05.966529 10625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:05.962932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47582 (local address 127.18.70.125:40561)
0806 02:17:05.963177 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:05.963181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:05.963263 (+ 82us) server_negotiation.cc:407] Connection header received
0806 02:17:05.963399 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:05.963401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:05.963450 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:05.963542 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:17:05.963960 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:05.964506 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:05.965390 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:05.965624 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:05.965734 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:05.966005 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:17:05.966080 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:17:05.966163 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:05.966284 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 02:17:05.966337 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":97,"threads_started":1}
I20260806 02:17:05.972926 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:05.969531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32884 (local address 127.18.70.124:39837)
0806 02:17:05.969767 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:05.969771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:05.969783 (+ 12us) server_negotiation.cc:407] Connection header received
0806 02:17:05.969835 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:05.969837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:05.969961 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:05.970052 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:17:05.970234 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:05.970723 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:05.971461 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:05.971743 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:05.971821 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:05.972084 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:17:05.972523 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:17:05.972616 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:05.972666 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 02:17:05.972712 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":78,"threads_started":1}
I20260806 02:17:05.979668 10625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:05.975684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47592 (local address 127.18.70.125:40561)
0806 02:17:05.975817 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:05.975820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:05.976310 (+ 490us) server_negotiation.cc:407] Connection header received
0806 02:17:05.976415 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:05.976419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:05.976471 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:05.976544 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:17:05.976774 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:05.977385 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:05.978506 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:05.978773 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:05.978889 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:05.979163 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:17:05.979261 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:17:05.979324 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:05.979447 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 02:17:05.979500 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 02:17:05.979770 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:05.976033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32886 (local address 127.18.70.124:39837)
0806 02:17:05.976179 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:05.976184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:05.976396 (+ 212us) server_negotiation.cc:407] Connection header received
0806 02:17:05.976566 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:05.976573 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:05.976633 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:05.976696 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:17:05.977136 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:05.977742 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:05.978608 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:05.978943 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:05.979037 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:05.979276 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:17:05.979347 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:17:05.979414 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:05.979586 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 02:17:05.979635 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 02:17:05.980613 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 02:17:05.975660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49868 (local address 127.18.70.126:39099)
0806 02:17:05.975982 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0806 02:17:05.975986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 02:17:05.976001 (+ 15us) server_negotiation.cc:407] Connection header received
0806 02:17:05.976344 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 02:17:05.976350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 02:17:05.976523 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 02:17:05.976641 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 02:17:05.977152 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:05.977857 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:05.978974 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 02:17:05.979312 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 02:17:05.979434 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 02:17:05.979767 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 02:17:05.980109 (+ 342us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 02:17:05.980223 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 02:17:05.980297 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 02:17:05.980351 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":80,"threads_started":1}
I20260806 02:17:05.998534 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 10248
I20260806 02:17:06.008833 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 10328
I20260806 02:17:06.016264 18713 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskVlzDnt/build/debug/bin/kudu with pid 10407
2026-08-06T02:17:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3708 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6840 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (286229 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST